Too often, we’ve made incorrect assumptions about our security. Fraudulent charges are covered by our credit card protections, while the FDIC protects our bank accounts. When the entire financial system collapsed mostly due to the subprime mortgage lending industry, we assumed we’d be okay again, and the government bailed out the banks.
We should never take our security for granted, and this is especially true when it comes to blockchain technology. Blockchain projects remain in the early stages, so it’s important to verify that the coding behind crypto wallets, exchanges, and projects are secure. No industry leader understands this better than Trail of Bits founder and CEO Dan Guido.
Guido’s firm specializes in security testing, as if they were hacking their own clients to find their vulnerabilities. Dan Guido’s exclusive interview with The Merkle is an opportunity for the crypto community to proactively address our assumptions about security and safety.
The Merkle: Can you give us a brief history of Trail of Bits and the scope of its projects?
Dan Guido: Trail of Bits has been around for almost seven years. I founded the company with my partner Alex Sotirov. We are both security researchers, and we’ve been doing this since we were fifteen years old. We don’t have any venture funding; we built the company from the ground up.
We work across many industries including tech, finance, and defense. We audit high-assurance financial applications code, low-level code, and cryptographic systems. We work on airplanes for Lockheed Martin, security operations software for Facebook, and security research for DARPA. Since blockchain emerged as a new technology, we have been able to apply all that experience to this new field.
The Merkle: What makes Trail of Bits particularly qualified to do security engineering and assessments on blockchain technology?
Guido: We’re a software security company, and that means we’re constantly working on compilers, binary analysis, programming languages, and trying to find software security flaws, sometimes without even looking at source code. We know what tools to write and what processes to construct. We can tell what good and bad code looks like because we’ve seen it all before; it’s stuff we’ve spent our whole lives on.
About two years ago, we focused on porting the tools we built [to test software and code in other industries] to blockchain technology, particularly the Ethereum Virtual Machine (EVM). Now we primarily offer three services to clients in this space: smart contract audits, design guidance for asset custody, and blockchain design.
For smart contract audits, we’re given a DApp – typically written in Solidity – and apply our unique set of tools and knowledge to help uncover hidden risks. We write new software test cases and provide guidance to help projects stay secure, even after our engagement is finished.
We also look at custody systems, as they are designed for exchanges like Gemini, ICOs, and organizations like the Web3 Foundation. For these projects, we’re designing and reviewing systems that access and store funds.
Finally, we also help with blockchain design. In one notable case, we worked with the RSK blockchain, which puts smart contracts into Bitcoin, and helped review their contract runtime environment. We have both theoretical and applied cryptographers who can do real assessments of blockchain design choices that many other companies cannot.
The Merkle: What are some of the blockchain projects you’re working on?
Guido: Specific to blockchain companies, Trail of Bits has worked with LivePeer, Golem, MakerDAO, and many others we’re not able to disclose. Code auditing isn’t new, but the rapid growth of smart contracts has created an immediate need for testing. From infamous hacks to failed exchanges to enterprising hackers stealing cryptocurrency, it’s clear this industry requires rigorous testing to prove applications work as promised and remain secure.
We started with only one engineer focused on blockchains, working on it out of interest. Today we have ten. Even with all those security engineers, Trail of Bits still has to be selective about new clients, and there are a lot of people we unfortunately turn away. We choose clients who build foundational technologies, take on risks, or who present us with interesting intellectual challenges.
The Merkle: Tell us about your work on Ethereum with fuzzing, particularly your EVM Smart Fuzzer, Echidna, released in early March. On your blog you said, “It’s the first-ever fuzzer to target smart contracts, and has powerful features like abstract state-machine modelling and automatic minimal test case generation.” What are the implications?
Guido: A fuzzer tries to violate assumptions about how code will act. In this case, we’re generating sample inputs to find unexpected problems in Ethereum smart contracts. Echidna is smart about what tricky inputs look like and can generate millions of test cases at a very high throughput to stress test smart contracts.
The potential inputs to a program could be vast, so a good fuzzer must be both really fast and really smart at finding which potential inputs are more effective at breaking the program than others. Echidna does both of these things.
If you’re working with typical compiled code like C++, then you’re looking for a crash. However, in Solidity or EVM bytecode, you don’t know exactly what a bad thing looks like. It could be a wallet drained or accessing someone else’s data. Echidna has an expressive language that lets you customize what properties it’s looking for in these cases.
The Merkle: So, essentially, it tries to make things that must always be true become false?
Guido: Yes, and Echidna tracks the amount of the code it has tested while it works. When it’s tested close to 100%, then it has tried almost anything someone could do to a program. It flails around like crazy trying to find ways to do things you don’t want, testing to see if it can make your application work incorrectly.
This kind of testing gives high assurance your program won’t do something unexpected, like lose all your ether. Echidna is best to use after you add a new feature. Write test cases for it and Echidna will do its best to break the code.
An Echidna test showing problems with Solidity coding.
The Merkle: When someone like Golem goes to you for a smart contract audit, what do you do?
Guido: As a starting point, we ask about the use, architecture, implementation, and testing of the product. Then, we ask about their nightmare scenarios. We’ll use that foundation to search for scenarios where they might become true. We meet with the engineers weekly to review what we’ve found, discuss potential fixes, and make sure we’re reviewing for the right issues.
This process typically takes two to eight weeks. At the conclusion, we write an audit report that lists all our high-level concerns in addition to the specific flaws we found. For example, are there systemic issues with how they write code or parts of the code base that should be checked later? What matters most is that they fix the code identified. In the final debrief, we want them to have the tools and knowledge to fully address all the issues.
The Merkle: Why are these audits so important?
Guido: The risk and consequences of failure when using this technology is high. Blockchain technology is very unforgiving. Transactions are irreversible and participants are pseudo-anonymous, which makes it easy for hackers to steal cryptocurrency with impunity.
Each new application has its own set of business risks too. For example, if you’re depending on a stablecoin not changing value, yet someone can manipulate its price ratio on demand, then that is a security flaw that could let someone make millions. We have to deeply understand each project we work with to find these application-specific flaws.
The Merkle: Yes, the recent phishing attack on MyEtherWallet is yet another reminder of hackers’ ability to steal funds in this space. What steps do you recommend for securely developing smart contracts?
Guido: Many developers rush into writing Solidity because it looks like JavaScript and that makes it easy and familiar. Before you begin, I recommend closely reading the Solidity language documentation and our “Not So Smart Contracts” reference to learn from others’ mistakes. The language, and this whole field, is a work in progress, so it pays to understand its foundation. As you’re writing code, use the best tools available to ensure that each line is correct: use the latest Solidity compiler and review the warnings, write high-coverage unit tests, fuzz the code with Echidna, and symbolically execute it with another of our tools, Manticore, to verify it works correctly.
If you’re truly writing high-risk code, you should talk to an expert. Even if you’ve run through all the right steps, you need a professional, considering what is at risk. These are still the early days, and most of the development tools are not refined. We invest so much in tools to help make this easier for everyone to get right.
Bugs present in Solidity eliminated from other modern programming languages, from a controversial Trail of Bits presentation titled “Black Hat Ethereum”.
The Merkle: It sounds like, despite its popularity, there are some serious problems around coding in Solidity. Can you explain them?
Guido: Solidity has reintroduced bug classes we’ve mostly ironed out from other programming languages. There’s dozens of problems even languages like C, C++, Go, Rust, and Swift have eliminated, where Solidity is reintroducing them all over again. There’s also a financial cost to everyone when bad Solidity code is run in the EVM; it costs real money (in gas) to run inefficient code on smart contracts. I’m really anticipating a move to WASM (Web Assembly Stacked Virtual Machine).
If WASM replaces the EVM, it would let the community build tooling on LLVM (Low Level Virtual Machine). This would be a huge benefit since LLVM is a vastly more mature compiler toolchain, with support for many languages, optimizations, and analyses that Ethereum could use as well.
Regarding the longevity of the Solidity language itself, I think there was a clear benefit in the early stages of Ethereum to [using] a language built for easy adoption like Solidity. However, now that we’ve seen what’s possible, it’s time to consider a safer, more efficient, and more secure method to build smart contracts.
