STAPR
Smart Testing for APR Tools
About
The problem it solves # STAPR ### Smart Contract Testing for Automatic Program Repair In Automated Program Repair (APR), testing of patched code is always difficult. Usually, evaluation of APR tools is done via test suites; however, these are not perfect oracles. When dealing with APR of smart contracts, we have one advantage: The whole history of calls for all contracts is stored on-chain. This means we can extract all call data for any given contract, and replay it on a patched version. This serves two purposes: (1) we can make sure that the patched version of the contract does not break the correct logic of the original contract, and (2) we can observe if the patch actually fixes a given fault observed at a given block. ------------------------------------------------------- STAPR is a tool that automates and collects the call history of smart contracts using TheGraph. It does so in the following steps: - Given a smart contract address and creation block - It fetches the contract's source code - It parses the source code and retrieves all callable functions - It creates a subgraph skeleton via `graph init` - It modifies the subgraphs yaml definition, adding automatically one CallHandler per callable function - It generates the handler functions that store the call data as entities - It rebuilds the modified subgraph via `graph codegen` and `graph build` Deployment of the subgraph is done manually via `graph deploy` after creating the subgraph in the subgraph studio. STAPR is also able to replay the call history for patched contracts: - It fetches the call history of the original contract by using `graphclient` - It deploys the patched contract in a local test network using Foundry/Forge - It inserts all the call history in a test scenario - It runs the test and validates the invariants: the patched contract's behavior does not diverge from the original, and the fault which is the fix target does not occur.
How it's made
Challenges we ran into As researchers, initially, we aim to automate the process for a significant dataset. Sadly, selecting relevant real-world contracts, with known attacks with verified source code was very complicated. Curating such a dataset would have consumed most of the time on the hackathon and there we narrowed our experiments towards one contract. More difficulties were related to the technology, none of the team members used TheGraph or FOundry before this hackathon. Regarding using TheGraph we have a more overall conversation with our mentors regarding the limitations of the tool. Still, we did not get help on the code.