What the heck is Bitcoin Core thinking?

Yesterday, Eric Voorhees made a post on Reddit titled "IMHO BTC price will be weak until Core demonstrates competency in social consensus.". While I don't necessarily totally agree with this, I do see his point that Core (or other people that share the view of Core) can improve their communications and help build social consensus.

This blog post will be my attempt to help explain the thinking of Core. While I'm not a Bitcoin Core developer myself, I have been following this space as a Bitcoin holder for quite a while and also have 15 years experience as a software developer and software development manager. I believe I have an understanding of what the Bitcoin Core developers are thinking and will attempt to convey that.

In my experience as a developer, very frequently a product manager will come to you and say, I want you to implement X. If X is something technical, the next question is: why do you want me to implement X? In some cases, implementing X is the right solution, but in other cases it's not the best way from a development standpoint to achieve the goal that the product manager is attempting to achieve. Throughout the years, one of the frequent statements that I make to product folks is: "Just give me the requirements, don't get into implementation details". The reason that this statement is frequently useful is because it allows the product manager to focus on what they actually want and avoid dictating how to do engineering work. Engineering work is the task of the developer, not the product manager and we are trained to do it better. It's also our obligation to explain why we intend to implement the requirements the way we intend to implement them.

I believe that the Bitcoin ecosystem is focusing too much on implementation details and not on product requirements. What users are saying is: "I want a bigger block size now!". The block size is an implementation detail, not a requirement. The requirement is actually that users want to continue to have low fees. Users are also worried that Bitcoin's throughput will not keep up with demand for transactions. These are valid requirements, but the requirement is not to increase the block size, that's the implementation detail. In addition to the requirement that transaction fees stay low, there is the other requirement that Bitcoin remain decentralized. That requirement is more important than low fees because without decentralization, Bitcoin is no real different than legacy currency systems and thus has no real value.

So, given these real requirements (low fees for an increasing user base, and decentralization) we now leave it up to the engineers to come up with the solutions to these problems. That's what the two scalability conferences last year were all about. So, the engineers got together and discussed it and the solution they came up with is multi-fold:

SegWit

Lightning Network

IBLTs and Weak Blocks

A block size increase after these things are implemented A full technical discussion of the Core roadmap can be found here.

So, lets break this down. What do each of these things do? First SegWit. SegWit is a method to separate (Segregate) out part of the Bitcoin transactions called the signature (Witness). What this does is allow us to discard some of the data (the signatures) in blocks that are only needed at the time of validation over time. The benefit of this is that we can scale Bitcoin without some of the downsides of increasing the block size. The net-net of this is that we can expect somewhere between 1.7x to 2x increase in the number of transactions to fit into blocks based on the current proposal. As mentioned in the roadmap, this change is expected to be available in April, 2016. Now, this change alone does nothing, because wallets need to upgrade their code to support this change. But, several wallets have stated that they intend to roll out this change concurrently with the release of Bitcoin core's code. There is some analysis on how much of a cost reduction will be generated by SegWit that one can view here. So, once again, when we focus on what customers want (fees to remain low), all that one needs to do is upgrade to a SegWit enabled wallet (which should be available in April) and you will get somewhere near 50% reduction in cost of transactions and we're talking about a few months out.

Next, we have Lightning Network. There is a lot of technical detail to Lightning Network, but to summarize what it does is that it allows for effectively the same level of security of the Bitcoin blockchain without actually submitting most of the transactions to the blockchain. With this method, even with the current 1mb limit on block size, lightning network could allow 50 million users to do an unlimited number of transactions for very low fees. Full detail here. As seen in the link, modest increases to the block size would multiply this number such that vast number of Bitcoin users could engage in unlimited number of transactions. One of the additional benefits of Lightning Network is that payments are irreversible instantly and we no longer have to wait for multiple confirmations once you have your coins in the Lightning Network.

Next we have IBLTs and Weak Blocks. IBLT is relatively new data structure that allows for somewhat lossy data transmission. It was initially proposed for use in the Bitcoin network by Gavin Andreesen here. The basic idea is that you can compress the data into this new data structure and reconstruct the data even in the presence of partial loss of data. This is useful in the Bitcoin network because that's exactly what we have a network where some (or all) transactions may not be propagated to all nodes. Thus an IBLT allows for blocks to be transmitted much more efficiently and potentially cuts bandwidth in half. Weak blocks are a method for allowing miners to pre-send the blocks that they are working on so that when a "strong" block is discovered, the network generally knows which strong block was won and it has propagated through the network faster. These two techniques will allow for a massive improvement in bandwidth consumption of Bitcoin and when they are ready, a hard fork to increase block size could be implemented.

So, what does all of this mean? With the combination of these changes, we are looking at a MASSIVE increase in capacity of the network over the next 1-2 years which will keep transactions cheap and maintain decentralization. As I pointed out, those are the customer requirements, NOT a block size increase. That's the implementation detail only. There are also other technologies that are not included in this summary (e.g. Schnorr signatures, possibly technologies/combinations of IBLT/Weak blocks/other tech). This vision is actually much more ambitious than even a large block increase like BIP101. Even if we were able to support much larger blocks we couldn't support all the worlds transactions. Some have argued that we could come close with BIP101 (which may not be technically feasible), but I think this view is limited to something like replacing the VISA network. While replacing the VISA network sounds ambitious, if you look at all the worlds transactions (including cash payments, stock trades, etc), the VISA network is actually relatively small. With the technologies discussed in this blog entry and the Bitcoin Core roadmap, we could see a much more ambitious goal and really replace the vast majority of all transfers of value in the world. That would make the world a better place and meet the requirements of the users.