Roundup Q2 - Ethereum Blog

Thanks to all the developers and team leads who contributed to the sections on their projects

In the last month and a half, the Ethereum network went through a rapid growth in usage, to the point that it now processes as many transactions per second as Bitcoin. To accommodate the increased load, which has on a few occasions reached the network’s full capacity for hours at a time, the community independently came together and miners voted to increase the gas limit to 6.7 million. We at the Foundation have been rapidly putting additional resources toward increasing the efficiency of the network, alongside planning longer-term changes that will greatly increase the network’s scalability.

Research

The pyethereum client has seen a substantial revamp, and version 2.0 has been released. See https://pypi.python.org/pypi?:action=display&name=ethereum&version=2.0.4 for download; in Ubuntu you can also do “sudo pip install ethereum”. Additionally, we have implemented experimental versions of:

Metropolis testing

Metropolis testing is rapidly moving forward. We are actively seeking additional help with finishing testing. See:

We have started a substantial cross-client benchmarking effort to identify places that are in greatest need of performance improvement. See some preliminary results for opcode benchmaking in geth here: https://gist.github.com/holiman/7b157e4074707f6ba3196d46ad3745af

Ethereum core developer meetings #15-#19 took place. Notes and audio/video of the meetings can be found here: https://github.com/ethereum/pm/tree/master/All%20Core%20Devs%20Meetings

Mist team

In May-June, the Mist team had a team meetup: for one week the team had face to face meetings, some members for the first time, in which we sat together to share details on projects we were working on and to talk about the current codebase and future roadmap. While we have a long list of features we are working on, we realized that most of the issues reported on github were related to two main issues: slow synchronization and lost account private keys/passwords. We outlined features that we could implement to help prevent user errors and other related issues, including more options for node switching (including Infura support) and better options for account management (including HD wallets and mnemonic seeds – but with a twist).

Many of those new issues require some changes on how the signing process is done to make Mist more independent of Geth, which is being worked on as a standalone signer. We have also done some research on refactoring parts of the Mist codebase to make it more modular and easier to maintain. Victor Maia presented some research on how to make apps load quicker and be more reliable and we are currently testing some of these concepts as either pieces of the main codebase and/or an alternative web-based product. Progress has been made on ENS integration: we have added ENS support to our address component, meaning that any of the apps we have built in meteor (wallet and ens registrar app) will accept a name in any field where it would usually expect an ethereum address. We are also working on making a web component for generic input types for ethereum addresses, so any webapp developer can use an input field with support for ENS, checksum and ethereum identicons. With swarm now using the main net registrar, it also means that Mist will accept ENS addresses on the url as soon as the swarm branch is merged.. Swarm integration has been tested and is a lot more stable than it has been even a few weeks ago. We predict it will finally be ready to release soon. Web3.js

Web3.js is  coming along well. The new whisper API was recently added to the old 0.x.x and the new 1.0.0 version. Whisper v5 is currently only available in geth and needs to be started using –shh.  We are currently adding swarm.js and finishing the JavaScript account management. If everything goes well, an alpha release will happen soon.

You can already test the new web3.js 1.0 here: https://github.com/ethereum/web3.js/tree/1.0, and read the new docs at http://web3js.readthedocs.io/en/1.0/

Security

We have received several bounty submissions for vulnerabilities in EthereumJS, Solidity and Cpp-ethereum. See the leaderboard for the current stats.

We now have pyethereum on board on the cross-client blackbox consensus testing in Hive, which continuously performs over 10K tests on each client. See http://hivetests.ethereum.org. As a lightweight alternative to Hive, we’ve also started a project to perform fuzz testing directly on the virtual machines, starting with Geth, Parity and Python. In the same vein, we’ve also set up an automated AFL-based fuzzing of Solidity.

In preparation for Metropolis, a benchmarking suite for the Geth EVM has been implemented to ensure that the gas prices for new opcodes and precompiles are within reasonable bounds, so as to not constitute DoS-vectors at a later point.

EVM 1.5

The “EVM 1.5” proposals are now EIP drafts for “Subroutines and Static Jumps for the EVM #615,” and “SIMD Operations for the EVM #616”.  Discussion and critique are welcome at the https://github.com/ethereum/EIPs/issues/615 and https://github.com/ethereum/EIPs/issues/616 conversations.

EthereumJS

The ethereumJS team is still looking for community contributors to help the Ethereum Javascript client reach Metropolis compatibility. A video walkthrough “Intro to Core Development with Ethereumjs-vm” has been released.

Light Client

New algorithms have been designed and implemented in order to improve log searching performance in the next version of the LES protocol. Promising R&D work has been done toward achieving quick and trustless initial syncing without hardcoded checkpoints. We have put some efforts into finalizing the topic discovery protocol, which helps clients to find suitable LES servers as it is currently a somewhat weak point in the experimental light client service.

Remix

The main Remix feature in the last month is the alpha release of Remixd: https://github.com/ethereum/remixd

Formal Verification

The progress in the eth-isabelle project was mostly from external contributions. In particular,  the better separation logic tactics, which were externally contributed allow much shorter proofs about Ethereum contracts.

Better separation logic tactics (contributed) Coq build fixed, and added in the continuous integration Removing unmaintained files, and `Annotation` structure not needed anymore (PR pending) Running Blockchain Tests (in progress; ecdsa recover implementation in OCaml wanted). Bamboo The compiler is generating bytecode for all initial examples Syntax perfection following community feedback End-to-end testing of the compiler (in progress) Solidity

The Solidity project has been quite active in the last months, although most of the updates is not yet directly visible on the user side. We saw a lot more engagement by the community and now have volunteers regularly contributing both to the core code as well documentation including translation, mainly into Spanish.

 

We added a feature that allows export of the full abstract syntax tree with all type annotations, which makes it much easier to write tools that would otherwise need a custom-made parser. The next feature will be to also re-import this data after potential modifications, which would allow things like mutation testing.

We extended the inline assembly language with structured elements (for, switch and functions) and deprecated manual jumps. This new inline assembly language will become a new intermediate language (together with type information), which allows Solidity to be more transparent in its compilation, more efficient (we will be able to add much more sophisticated optimizer routines) and more portable (it can already compile to EVM, EVM1.5 and some eWASM). We are currently rewriting the ABI encoder in this intermediate language which will include structs and nested dynamic objects.

Finally, we are adding an automated compile-time overflow and assertion checker which will also feature an SMT solver for the more complicated cases. Oh and of course, we have worked on tons of bugfixes and smaller features.

Swarm

The swarm team has onboarded new members and held an in-person Swarm Summit in Berlin in June, 2017. The week-long event brought together Ethereum team members, community contributors andspecial guests representing projects and companies interested in swarm. More than twenty talks and tutorial sessions were recorded. The edited videos will be published soon on the swarm summit website. Our public alpha test saw a great community response allowing us to gather more information on prospective user base needs and what the typical usage patterns might be. The high churn of nodes requires an explicit distinction between nodes that can and cannot commit to being available for a longer period of time to provide sufficient storage and bandwidth resources for the network. To support noncommiting nodes and mobile clients, swarm will offer various light modes of operation.

We have developed a suite of example applications highlighting the architectural and implementational peculiarities of Swarm-hosted distributed web applications that are quite a departure from the traditional client-server model. In particular, the building blocks of a distributed functional equivalent of dropbox are being developed, such as a web-interface providing a file-system view of swarm-hosted volumes, ENS integration, Mist-integration, FUSE mounting of swarm-volumes and privacy protections.

We added a new protocol, pss (bzz whispered) allowing internode messaging with deterministic routing based on the relaying kademlia network topology of swarm. The protocol uses whisper envelopes and offers udp-like protocol communication between nodes that are not directly connected.

Furthermore, we have developed a network testing and simulation framework in which we can model and benchmark a broad range of scenarios emerging from the interaction of a potentially large number of  nodes. This framework includes both scripting and visualization capabilities.

In cooperation with the Remix team, the implementation of a fully distributed integrated contract development environment is underway.

The next major release, POC 0.3 is scheduled to come out around Metropolis and will include  obfuscation support for plausible deniability, a rewrite of the swap peer-to-peer accounting for bandwidth incentivisation among other things.