Summary
The Kolibri Protocol has a component called the “Kolibri Liquidity Pool”, where kUSD holders can pool their kUSD together in order to liquidate larger ovens and split the seized collateral ratably among participants.
Unfortunately, early last week, the Kolibri Liquidity Pool experienced a sophisticated price manipulation/economic exploit. The Liquidity Pool’s code functioned correctly, but due to the economic conditions manufactured by the exploit, a large portion of the Liquidity Pool was siphoned away causing a loss of funds event for participants in the Liquidity Pool. The exploiter has returned all funds to pool holders, and there were ultimately no losses from this incident. The Kolibri DAO patched the Liquidity Pool in Kolibri Governance Proposal #12 and to our knowledge no funds are at further risk of exploitation.
Importantly, the Kolibri community has no reason to believe that the core Kolibri Protocol has any bugs or exploitable economic attacks that can be levied against it, and kUSD remains properly backed.
This post serves as a high-level overview of the incident and a brief primer on what occurred, aimed at a broad audience. Technical details on the vulnerability and a technical study of the exploit will be released shortly as separate posts. If you want to jump in early, feel free to look at our analysis dashboard.
Incident Details
On Dec 20, at 14:27 UTC the Kolibri Liquidity Pool had a sophisticated price manipulation/economic exploit run against it in this transaction. The exploit was very complex, with the operation group consisting of 14 independent transactions executed atomically, generating 66 contract calls. This exploit required 642,000 XTZ and 63,742 kUSD to execute successfully (about $2,573,962 USD in value at exploit time, with the TWAP from the oracle at $3.91/tez).
The exploit was made up of several individual transactions that were executed atomically by the Tezos network, which Tezos refers to as a “batch” of transactions. When executed, the exploit batch manipulates the Quipuswap kUSD/XTZ price with 4 separate (large) trades, then executed 2 liquidations of very large under-collateralized ovens through the Liquidity Pool, which in turn activates the Liquidity Pool to sell the seized collateral (a total of 741,199.57 XTZ) into a highly manipulated market where 1 kUSD was effectively worth ~$19 worth of tez, trading the seized collateral for a measly 75,884.47 kUSD.
The transaction then arbitrages down the Quipuswap pool manipulation to return the price of kUSD back to $1 by buying the recently sold collateral for a massive discount and profiting both from the liquidation of the ovens while also having the Liquidity Pool pick up the tab for the loans, draining ~1.5M kUSD from the pool in the process. Ultimately, the exploiter exchanged 642,000 XTZ and 63,742 kUSD for 1,307,586 XTZ (net profit of 665,586 XTZ or roughly $2.6M at $3.91 tez).
Thankfully, a day later at 22:55 UTC, tz1PnbhBb2TV4tJjX1BpAScx8rPk2YAcc5VD, the exploiter, sent a batch of 218 transactions returning kUSD to participants in the Kolibri Liquidity Pool, which effectively reimbursed them for the rewards (and then some).
At the time of exploitation, The Liquidity Pool held 1,663,714 kUSD, and the exploiter returned 861,089 kUSD back to 218 pool participants (across both farmers and individual QLkUSD holders), at an exchange rate of 1 QLkUSD = 1.52 kUSD (the pool rate was 1 QLkUSD = 1.48 kUSD at time of exploitation). Interestingly, the exploiter didn’t repay 2 addresses with large holdings in the pool (worth ~730,000 kUSD at time of exploitation), but has moved funds between these addresses seemingly in coordination, implying these addresses are in control of/working with the exploiter (or exploiters). These addresses are also all quite active within the Kolibri products, farms, and even vote on governance proposals.
Worth noting as well is that the exploit didn’t completely drain the pool, instead leaving it with 172,916.86 kUSD, creating a new redemption rate of 1 QLkUSD = ~0.15 kUSD. For example: if you would have deposited 10 kUSD on the block before the exploit (exchanging it for 6.76 QLkUSD), the exploiter sent you 10.28 kUSD, and you could redeem a further 1.01 kuSD, ultimately leaving you with 11.29 kUSD (an approx. 13% gain).
Therefore, based on Hover Labs’ and the Kolibri community’s analysis, we believe this to be a full reimbursement of all positions in the liquidity pool, and that it’s likely that the exploiter (or exploiters) were simply using the Kolibri Liquidity Pool as a means to liquidate the large uncollateralized ovens, and that the Liquidity Pool itself was (for a lack of better term) collateral damage.
Shortly after the return of the funds, Hover Labs submitted KIP-017 and associated DAO proposal, Kolibri Governance Proposal #12. This proposal removed the Liquidity Pool’s ability to liquidate ovens, which makes the economic exploit impossible while the community and DAO figure out the next steps for the Liquidity Pool. This proposal was approved by the DAO, and after being timelocked, executed on December 27, at 13:50 UTC.
Vulnerability
Participants in the Liquidity Pool who pool their kUSD together in order to liquidate ovens provide a crucial function within the protocol. When the pool liquidates an oven, it pays off the uncollateralized oven with the kUSD of participants and receives the oven’s collateral, which is XTZ. Bookkeeping a pool of two assets is a difficult task (because of the computational constraints of algorithms and data structures on a blockchain) so to simplify this flow, the Liquidity Pool simply swaps any received XTZ for kUSD on Quipuswap. Quipuswap was chosen because it is the oldest and most liquid decentralized exchange for kUSD.
The swap occurs as a market buy, such that the amount of collateral in an oven is traded for kUSD on Quipuswap, regardless of the current market price. For large ovens, this means that a large buy will be executed at an unknown price, likely with slippage. It’s well documented that Quipuswap has a 33% circuit breaker (trades on one side can’t drain more than 1/3 of the pool on the other) to prevent large amounts of slippage, and while it’s possible to manipulate a pool with this sort of protection, you would need to manipulate the pool atomically (all at once) to be able to force a losing trade (with anything <50% slippage, the trade is still profitable).
Detailed Breakdown
A detailed breakdown of the exploit step-by-step (and the clever math that makes it work) is planned for an additional blog post, but you can find a walkthrough of the exploit here, which provides the actual numbers used as well as simulations for cost-of-exploit analysis.
How Did This Happen?
Vulnerabilities resulting in loss of funds events are as bad as things can get for a DeFi protocol. Depending on the nature or impact of the vulnerability it can shake trust in the protocol soundness, development team, or even the broader DeFi ecosystem the protocol sits in. Kolibri is one of the oldest DeFi products in the Tezos ecosystem, and takes security very seriously.
Building new and novel things on greenfield technology has a ton of things that can go wrong — the tooling is all generally brand new, and subtle nuances to how specific aspects of this technology function can have dire consequences, especially when there’s strong economic incentives to discover and exploit these bugs/nuances/vulnerabilities. Ultimately it makes for a more secure ecosystem overall, but it does mean pain along the way.
Combined with a massive scheduling backlog (and expenses!) for security reviews, the unfortunate reality is that to launch new features presents a dilemma — do you ship new features that haven’t been reviewed formally, or do you wait for reviews to be able to launch? We think a happy middle ground is to launch features that are un-reviewed, but make it abundantly clear that the feature is unreviewed (something not often seen in the defi ecosystem).
When users visit the Kolibri Liquidity Pool page, display a click-through warning that the component you’re interacting with has *not* undergone a security review, and link you directly to the code so you can read it yourself.
Additionally, not only do we have a comprehensive protocol risk doc, the documentation on the Liquidity Pool itself calls out a distinct risk of economic attacks that we believed to only be theoretical and impossible to execute in practice (which obviously proved not to be true). The source code for the Liquidity Pool (and all Kolibri smart contracts) is actually completely open source as well, which is another thing not seen prevalent across the space. Having things be open-source means more trust in the protocol, but it also makes discovering security vulnerabilities easier and more practical.
One unique (and amazing) aspect of Tezos is its ability to do core protocol upgrades, and change how the platform that runs smart contracts functions, which can be both a blessing and a curse. The Kolibri Liquidity Pool was deployed in April, when the Tezos network was running the Florence Protocol. At the time the gas costs to execute the %liquidation entrypoint was 80%+ of the operation gas limit of 1M gas which meant it was unfortunately not possible to add better safety checks and the pool would have to instead have to rely on the 33% slippage protection built into Quipuswap itself (with a 200% collateralization ratio requirement on ovens, any trade that has less than 50% slippage is profitable).
Tezos is special in a number of ways, but one of its interesting features is the ability to not just execute single transactions atomically (so it, and all sub-calls all succeed or all fail), but you can also batch individual transactions together to be executed as a single atomic block, one after another. Many know about this feature because it’s very commonly used when sending tokens to bundle the “approve” transaction along with the transfer request.
Unfortunately, we had a simple (mis)understanding that the 1M operation gas limit applied to the entire batch of transactions (it does not) instead of just each individual transaction in the batch. As a result the practical price manipulation attacks we thought were impossible, needless to say, actually are possible, and much more practical than originally theorized.
Next Steps
Now that the vulnerability is patched we’re going to release our detailed analysis and findings on what was an extremely interesting and quite novel exploit. To our knowledge this is by far the most sophisticated exploit ever performed on the Tezos blockchain. This is a very strong indicator that the ecosystem is maturing, and that actors on-chain are growing more sophisticated. It’s nowhere near the “dark forest” of Ethereum, but to us this signals an end to the “safe times” wherein simpler vulnerabilities were able to exist for periods of time without exploitation, even with substantial funds at risk.
Regardless, We think we can speak for the entire Kolibri Protocol participants and DAO in that we are eternally grateful to the exploiter for returning the funds drained from the pool and making us aware of this security vulnerability. Ideally it should never have happened, but in the spectrum of outcomes possible, this was the happiest possible ending.
In the immediate term, the Liquidity Pool is currently disabled and unable to liquidate ovens, meaning that the Kolibri Protocol temporarily has a slightly lower minimum collateralization ratio for ovens (180%), so the focus is about what to do with the Liquidity Pool moving forward, with a few different paths possible (ranging from abandoning the Liquidity Pool entirely to an interim fix until a better v2 can be developed). If you have opinions please join the Kolibri discord server or post to the Kolibri forum.
Medium term, this incident likely spurs community prioritization of the Kolibri Bug Bounty Program, which is an idea that has been discussed a few times but was deprioritized in favor of shipping features like the Kolibri Savings Rate. There are a number of challenges when it comes to running a security bounty/vulnerability reporting pipeline in a decentralized way, but likely this will lead to the formation of a specific Kolibri Security Council to triage and help handle anything security related.
Longer term we have the Kolibri DAO slated for a full security review kicking off in March (thanks to the TF for sponsoring it), which is not just the final phase of our progressive decentralization plan, but once complete will usher in a new phase for the Kolibri protocol full of new and interesting things!
