How to scale Bitcoin to VISA levels without a hard fork or blocksize increase

In a presentation last week, Adam Back, the CEO of BlockStream, talked about Bitcoin scaling tradeoffs. At just about exactly one hour into his presentation, Back discussed a slide entitled "Future Scale Sketch (my opinion)". In this slide, he gave his take on how Bitcoin scaling should be implemented. He mentioned that Segwit will increase capacity of Bitcoin by approximately 2X and he also proposes implementing Schnorr signatures which would give an additional 1.5X - 2X capacity increase. Both can be implemented as soft forks. Combined, these two improvements would account for a 3X - 4X increase in throughput. Back also went on to discuss a potential hard fork increase and capacity and the lightning network, but we'll get back to that in a moment.

In an article in Bitcoin Magazine, entitled "The Power of Schnorr: The Signature Algorithm to Increase Bitcoin's Scale and Privacy", the author, Aaron van Wirdum, discusses how the Schnorr signature algorithm can potentially improve both privacy and scale of bitcoin at the same time. Because Schnorr signatures allow participants in CoinJoin to combine their signatures into a single signature thus greatly reducing the amount of space needed to engage in a coinjoin transaction on a per participant basis. This would require a CoinJoin protocol (possibly something like the join market) to be widely adopted by most wallets, but is certainly something that is within technological possibility.

With the reduced sizes by combining transactions, via CoinJoin and Schnorr, we should be able to increase capacity more towards the higher end of Back's estimates of 4X. Current capacity is about 3.5 transactions per second, so with these improvements, we'd be at about 14 transactions per second.

Now, if we go back to Back's presentation, he mentioned at the bottom of the slide that we can expect somewhere between a 100X - 10,000X capacity increase by the lightning network. Even at the lower end of this estimate, we'd get 1,400 transactions per second based on these estimates. According to VISA, they handle an average of 150 million transactions per day. That comes out to an average of 1736 transactions per second. So, even on the lower end of the estimates by Back, we'd be right near the average capacity per second of VISA. It's highly likely that the lightning network will allow us to reach or far exceed the capacity of VISA on an average basis, however we should also consider peak demand of the VISA network. According to the Bitcoin wiki scalability page, VISA's daily peak is around 4,000 transactions per second. While it's likely that lightning network will scale above that based on Back's numbers since that is still very much toward the lower end of his estimates for capacity increase, it's important to note that due to the way that the lightning network works, short bursts of transactions are not likely to hit the network. Only commitment transactions and the security based HLTCs (which are not likely to be needed very often) need to be broadcast to the blockchain. This allows for smoothing out of transactions throughout the day because commitment transactions can be delayed to a later date if necessary.

So, in summary, Bitcoin can and will scale to VISA levels without the need for any block size increase at all even at slightly higher levels than the lower bound estimates by Back and it's likely that it will scale much farther beyond those limits.

The one thing Bitcoin needs before it can be a success

There have been a plethora of blogs/news stories/articles/etc that have stated that Bitcoin needs to do ____ before it will be a success where ____ is anything from increase the block size, get regulated, better fungability, better exchanges, a marketing campaign, a new dev team, decentralization, more secure wallets, better scalability, BIP109, BIP101, BIP100, BIP9, Segwit, etc, etc. The list could go on and on but the reality is that Bitcoin can be easily obtained throughout the world and once you obtain it there are cheap methods to store it very securely. The truth of the matter is that only one thing needs to change before Bitcoin can be a success. Before I tell you what it is, I'd like to define what "success" is. For my purposes, success will mean replacing all forms of value transfer currently used in the world (meaning it replaces all fiat currencies and even gold, where it's used as a monetary instrument). The one thing that needs to change is our minds. What I mean is that as soon as everyone in the world understands the value of Bitcoin, it will replace all current forms of value transfer.

Some might say that this statement is ridiculous because Bitcoin only supports 7 tps or something like that. The reality is that if that's important to increase, it will be increased because there's enough value at stake to increase it or scale in some other way. The only thing that needs to happen for it to completely replace all money in the world is for everyone to figure it out. The basics of the system are already in place for this to happen because we have reliable exchanges around the world now. A rich person could wire $1 billion to Coinbase or Gemini or one of the other exchanges and start buying. But that's just speculation, right? Maybe, but why do people hold gold or cash? It's because it's a store of value. Bitcoin is an improved form of money based on the properties of money which are divisibility, portability, fungability, recognizability, scarcity, and durability. So, it is inevitable that this will be widely recognized.

One things to keep in mind is that this recognition is likely to happen VERY quickly. Many people talk about how Bitcoin might go up to $650 or $1,000 or $10,000 or even $100,000. The thing is when it's widely recognized that it is superior to form of money dollars will become obsolete. There won't be a memo. It will just start to happen and as the masses start to understand this, the value of all other paper assets will go to their intrinsic value which is practically nothing. Panic will set in and fear of missing out and KABOOM! So, the price of Bitcoin will go up to infinity because the value of the dollar, euro, yen, yuan, pound, ruble, etc will be zero. No one is going to notify you or give you a warning. It will just happen. If you understand this, you will be prepared.

There is an argument that has been put forth in the community that if transaction fees to up to $1 or $10 or something like that, Bitcoin will be useless. This is false. It doesn't matter what the fees are, if it's the only way to transfer value people will have to pay whatever fees are necessary and of course large holders would open up micropayment channels with each other and provide services for smaller users if this happened today. It would matter that this is not decentralized because all value in the world will be represented by Bitcoin.

All of this is inevitable. The only question is: when? For some reason, I have this sneaking suspicion that many people that have a lot of money understand this and are secretly accumulating as much bitcoin as they can while publicly smearing it. I suspect the price will start going up after the halvening. The real question is: how much will it go up at that time? Maybe it just goes up to a few thousand or so at that point, but if that's all that won't be the type of event that I'm talking about. When this event happens, all monetary value in the world will be transferred to Bitcoin. Maybe that's a few years later, but there is not doubt that the event is coming soon. Are you ready for this event? I am.

What are the consequences of this event? Of course, this event will be accompanied by mass hysteria because almost no one will know that Bitcoin is the new backbone to the monetary system when it happens. It will be proclaimed that no one could have possibly predicted this. But that won't be true. Some people will lose their life savings, but also everyone's debt will be wiped out so the majority of people will come out ahead. Society will benefit because we will be using sound money. The people who will be hurt are the people that have little debt and a lot of savings. Oh, by the way this event will cause all banks to become insolvent (as if they aren't already). Also, all bonds will be worthless because they are denominated in fiat currencies. Stocks that people own through a brokerage will largely be worthless as well because the brokerage holds many assets in bank accounts and also they do not hold all the stock certificates that they should. From what I understand the average brokerage holds about 20% of the stock certificates that they should and that means that when their bank accounts go belly up you won't be able to get your stock certificates (or maybe only 20% of them). Basically, you can assume that stocks will be worthless because others will be in line before you to collect any stock certificates that are left if this event happens. Of course, mutual funds, money markets, cds, savings accounts, checking accounts will all be instantly worthless. To summarize paper assets will evaporate instantly. Real estate will still have value but with this sort of collapse to the entire system home loans will not be available at all. When this happens, even if you have a fraction of a bitcoin, you will likely be able to buy up prime real estate. The holders of Bitcoin will inherit the earth so to speak.

This event will be epic and I don't see any way around it happening. It's not if, it's when. If you're not prepared, you should start preparing.

Why we should keep the Bitcoin block size at 1mb forever

In a previous blog post, I committed the heresy of heresies by stating that I think a fee event would be good for Bitcoin. In this post, I will go further and state my position that I think Bitcoin should keep the block size at 1mb (plus the additional allowance for signature data in the SegWit proposal) forever and never hard fork.

First off, lets discuss what kind of capacity we can expect with the current version of Bitcoin + SegWit + Lightning Network (the prereqs required for Lightning Network and SegWit are scheduled to be deployed in April of this year according to the Bitcoin Core roapdmap). According to Thaddeus Dryja's (the co-inventor of the lightning network) presentation at the Hong Kong Bitcoin scaling conference, with a 1mb block size, Lightning Network can support 8.3 million users. It's important to note that he also used a conservative estimate that only half of the capacity of the blocks would be used for Lightning Network transactions. He also did not include the additional capacity created by using SegWit (which since these are multisig transactions, could more than double the capacity). There are also things like Schnorr signatures which can further reduce the size of data and thus capacity. With all these improvements, we could easily see a 1mb support a capacity for 30 - 40 million users. Lets conservatively say that the Core protocol could have a capacity of 20 million or more users based on the changes being made this year. That is not to say that as soon as Bitcoin Core makes these changes, 20 million users will be supported because there is significant work to be done in wallets and layer 2 protocols to get us there. But, even if no additional changes are made to Core at that point, we can eventually support those users and if the block size does not change, there will be a financial incentive to upgrade the wallets and layer two tech. I will later consider Side Chains, but for now, lets discuss what the world would look like if, under no circumstances were we to change the block size limit.

So, 20 million users is great, but clearly we have more ambitious goals for Bitcoin. We want the entire world to be using Bitcoin. Right? Yes, of course, but the question is in what way? Before we answer that question, lets just look at what the picture would look like if the 1mb block size was kept. What would likely happen, assuming more and more people want to use Bitcoin? Clearly, Bitcoin would become a settlement layer only and off blockchain transactions would be required to scale. It would be similar to the current banking system where people use banks that settle with one another. That's bad right? We don't like banks right? Well, given what the banking system has done in recent years, this sort of reaction from bitcoiners to the notion of using "Bitcoin banks" can hardly be considered surprising, but lets look at this more realistically. First of all, these "banks" would not be anything like what we consider a bank today. They could (and would) operate behind the tor network (or similar) and be impossible to regulate. They would also be forced to compete with each other and given there's no way to regulate them, it would be a completely fair playing field. They would of course compete on fees and thus fees would likely go to 0. But, more importantly, they would compete on transparency. These off chain transaction providers (I will not call them banks) would be forced to cryptographically prove reserves at each and every bitcoin block. Greg Maxwell describes a method for doing so that, while ensures reserves for each user of these off chain providers, maintains their privacy as well.

With this model in mind, the capacity of 20 million users is much more than sufficient as that means we could have up to 20 million off chain providers that compete with each other which in turn provide services to the 7 billion people on the planet quite easily. That compares very favorably with the number of bank branches today. This document shows some interesting statistics about number of bank branches in the world. In the US, there were 32.4 bank branches per 100,000 adults in 2014. If we extrapolate that number across the world and make allowances for all people (not just adults), that means there'd need to be just over 2 million off chain providers to have a similar topology to today's banking system. This would also mean that each local branch was totally independent from each other which would lead to a much more decentralized system. While, this system is unlikely due to the fact that by being online this sort of locality is not necessary, it does illustrate that 20 million off chain providers is more than sufficient to create a vibrant market.

With the model described, we have a system that allows for 20 million interconnected off chain providers that can allow massive scale to the system. This system is not trustless, but any bad actor that maintained less than 100% reserves would be discovered within 10 minutes and completely lose all credibility for the rest of time. While exit scams would be possible, I don't see them as being practical because the cost of reputation loss would exceed the benefits gained by maintaining the trust. There would be fierce competition which would cause a race to the bottom in terms of fees, yet other services (like lending and insurance) could be provided and off chain provider services would be a loss leader. It would also be quite easy to diversify with many providers to reduce risks of exit scams. I also want to discuss the privacy implications, but before we get to that, lets discuss how this system compares to the modern banking system. In terms of financial inclusion, the described system is far superior because it can easily support the world's population. The modern banking system cannot. In terms of transparency, the described system is far superior because the off chain providers would be forced to implement proof of reserves which is impossible in the current system. Inflation would gradually reduce as Bitcoin's supply is limited, whereas in the current system, inflation is variable. Fractional reserve banking would be impossible in the described system because proof of reserves would make it impossible to implement.

Now, the remaining issue is around privacy. At this point, many will think the draw back is around privacy. Bitcoin's promise was in part to allow you to be your own bank and to hold your own private keys. Wouldn't letting someone else (like these off chain providers) hold your private keys mean you lose your financial privacy? First of all, as I mentioned, these off chain providers would be fiercely competitive and also be difficult to regulate (because they could run behind tor and other anonymizing tools and be in a diverse set of geographical locations). So, one of the things these providers would compete on is financial privacy. In the olden days, it was possible to have a "Swiss bank account" where your identity was not required and it was only secured by a number. But this type of privacy would again be possible in the described system, and it would be provided. But, that does not address the fact that at the off chain provider would at least be able to track what you are doing with your money. However, with advanced cryptography (like homomorphic encryption and zero knowledge proofs, and other things), one could envision a system where even the provider wouldn't know specifically who is authorizing the transaction, only that they are authorized to make the transaction. So, while research is required, it seems likely that the same level of privacy could be provided through the proposed system. Additionally, coinjoin across these off chain providers could be implemented as well.

With all this in mind, it appears that Bitcoin users would be able to get everything they want: financial inclusion, censorship resistance, sound money, transparency, trust minimization, without increasing the block size at all. The next valid question though is: why do this if we can just raise the block size safely to 2mb or some other number? Well, there are certainly some benefits to keeping Bitcoin the same and never having a hard fork at all. If, over a significant period of time, Bitcoin avoids ANY hard forks, that says a lot. It says that Bitcoin is almost IMPOSSIBLE to change. Since it works for certain things (and we can build many others on the layers above), that is a very good thing. If it were hard forking every few weeks, it might be harder to predict what Bitcoin will be say 20 years from now. With Bitcoin staying the same, innovation can move out to the layers above (much as what happened with TCP/IP and other technologies that have ossified). It also means that mining can be run through the tor network (or an internal version of tor that is built into the protocol in the future) and that it will be much harder to censor transactions. One of the great fears is that mining will be regulated, but if everything is run through tor, it would be that much harder to figure out where people are mining from. Another added benefit is that with limited space to store transactions, fees would go up, so there would be a force pushing fees to be sufficient to support the network as the block reward subsidy disappears over time.

The remaining thing to discuss is side chains. If the picture I painted is not sufficient to convince you that there's no reason to increase the block size, ever, then if you consider Side Chains, you might see that there really is no need to raise the block size. A side chain could easily be implemented with a much higher capacity. A safer strategy than increasing the block size on the main chain would be to add a side chain with a higher capacity. If problems occur with the side chain, we still have the main chain which has been unmodified. From a risk averse perspective, it makes more sense to take the risks on the side chains.

So, in conclusion I think the risks outweigh the benefits for a block size increase and I just don't see the need for it. The stability of a system that has never forked as enticing. Personally, I'd prefer to keep the block size at 1mb forever. If you disagree, I'd love to hear why and hopefully I've helped raise your awareness on this issue with my input. All the best!

Bitcoin and the immutable web

I've been thinking about what some of the next layer of apps that will be built on top of Bitcoin will be. Certainly, OpenBazaar is one of the more exciting applications and I believe has the potential to be the retail layer for ecommerce. Also, Storj has promise as a distributed Dropbox (although I believe it will possibly be replaced by a version that uses Bitcoin as the currency as opposed to it's own native token). Something that has yet to have been discussed much is what I will refer to in this blog as "The Immutable Web" (or unchangeable web). It's possible that the DATT project will turn into something like what I'm describing, but right now, they're describing the project as "A community with great content, where people get paid for their work.", which, while interesting is not exactly what I am going to describe here.

What do I mean by the immutable web? Well, to understand, lets say you had a piece of data, possibly text data, but any type of content will do, and wanted to ensure that people 1000 years from now were able to see it. How would you do it? Well, it would actually be quite hard to do reliably with today's technology. In particular, it's been very difficult to do this throughout history because during wars, the victors frequently destroyed libraries and other "content" throughout history. One of the often cited cases of this is the destruction of The Library of Alexandria. Some might argue that that doesn't happen anymore, but that's not accurate. Unfortunately, it is still the case that data is being destroyed by those that don't want information to be free. It's tempting to argue that this would not be the case when storing data with a popoular service like Facebook, Google, etc. For instance if you were to post a video on Youtube would it remain there for ever? Well, there's no reason to believe that data would be preserved and in fact, it seems unlikely to me that it would be preserved for extended periods of time. Also, it's definitely impossible if you violate any of the terms of service of Youtube, not just now, but into the future. For example, if you put bumper music in the background that someone claims to own, your video will be taken down. In particular, far into the future what if someone claims some kind of infringement after your death? What if someone doesn't like what you said or doesn't want people to know about something you said? To make a long story short, it's not something you can count on at all and I think it's highly unlikely that data on the web can be preserved, unless we create the immutable web.

Ok, I will answer my own question about what the immutable web is without giving any more examples. The Immutable Web is a decentralized distributed database of all content types which through the use of encryption and cryptocurrency allows for an incentive for storage providers to host content and protects that content by distributing it and hiding the actual location through encryption techniques (such as onion routing).

To be clear, I'm not talking about paying content creators in this case (or at least not directly). I'm talking about a distribution system for that content that offers proper incentives to host content and ensures any data that is put into this system permanently stays in the public domain regardless of whether or not anyone else finds that content to be objectionable.

So, how do we do this? Well, the main prerequisite that hadn't been around until recently was Bitcoin. Before Bitcoin, such a system would have been impossible because the incentive structure would be impossible. Now, that we have Bitcoin, we can setup such a system. So, how do we build it? It is a complex undertaking, but I see it as a merging of Bitcoin + Bittorrent + I2P. I'm not saying that it will actually necessarily use those technologies directly, but it will be an amalgamation of them. If we had a merger of these three technologies, you'd have a situation where consumers pay storage providers for the download of data in such a way where the consumer (or any observer) actually has no idea where the data is being hosted (due to the onion routing). With Bittorrent, we have this today with very popular content. For instance, if you want to download a very popular song or video, it will generally be available, because people are always downloading popular content and therefore, if at least 10 people or so are always downloading a file, it will always be available as these 10 people will be transferring chunks of the files to each other and onto the next 10 users as they intermittently join the swarm. The problem comes in with rare or less popular content. If you have a file that someone only downloads once per day, it's very likely that that file will not be available because once one downloads a file, one frequently disconnect from the network to avoid giving away free bandwidth or being discovered as participating in the network. The way that Bittorrent works is leachers download content from seeders in small chunks. Once a leacher downloads a chunk, it becomes a seeder for that chunk. Clients prefer to send chunks to other clients that have seeded recently so it's a system that incentivises cooperation. The problem is that once you have your file, there is no incentive to participate in the network further because you don't need anything. That's where Bitcoin comes in. If we could replace this preference algorithm with Bitcoin as the incentive (in the form of extremely small micropayments), there'd be an incentive to stay on the network and host content that is profitable. This would create a free market for hosts to store files to consumers. If you think about it, this is essentially what Youtube is doing, but in a decentralized way. They're hosting and serving all this data in exchange for serving ads to consumers. It's profitable for them to host even unpopular content that gets only one or two views. If we replaced this system with something like this Bitcoin + Bittorrent + I2P combo that I'm describing, we'd have a distributed version of Youtube which would be much more efficient, distributed, cheaper (since viewing ads is not free), and perpetual. Since hosts would only make money off of content that people actually download, there might be some content that eventually disappears, but since Youtube has a very long tail, I believe this same sort of long tail would occur in the system I'm describing as well. Also, unlike Youtube, such as system would be censorship resistant. There would be no way to enforce things like the DMCA and users would be free to mash up content as desired. Systems to flag duplicates and spammers would also be possible. Obviously, non video content would be able to work in this system as well.

One of the other interesting features of such a system is that all content could be timestamped into the Bitcoin blockchain so you'd know, provably, exactly when content was uploaded and you'd also know that it was not tampered with. Effectively, once you upload data into this Immutable Web, it would permanently be available in it's initial state. Data loss would be eliminated.

What would a system like this look like to users? It would probably look much like Youtube.com or Blogger.com, but it would be open source so different clients with various UIs would be available. It would be a gateway to all content available through all time. A business similar to bitcoin mining would form around efficiently storing copies of this data around the world. Incentives for low latency could be put into place to encourage decentralization of data as well.

The societal impacts of such a technology would be enormous. As they say knowledge is power, so now we'd put the power of all the world's knowledge into the hands of everyone. Some may be concerned that it would be impossible to maintain intellectual property. This may be a somewhat valid concern, but effectively, even with Bittorrent this is the case. From what we've seen it's just changed the way that content producers make money. While the music industry has lost revenue that they once made through CD sales, they can still make money off of live performances, merchandising, endorsements, etc. Bitcoin will also enhance the ability for musicians to do crowd funding and get donations. To me, it appears that the music industry has suffered greatly, but individual artists with talent have had an easier time to rise up through all the social media that is now available and it looks more like we're just cutting out the middlemen than hurting the artists. The same will apply to video as well.

So, in conclusion, the Immutable web will allow mankind to aggregate and distribute all the worlds content at extremely inexpensive commodity prices that reduce at the pace of moore's law. It's an exciting opportunity and it will be interesting to watch.

Bitcoin: Why a fee event would be good for Bitcoin

What is a fee event? As originally defined by core developer Jeff Garzik here, a fee event is an event where Bitcoin's blocks permanently fill up and thus some transactions must be rejected. As of today, there has not been a fee event because blocks are of variable size and over time, even transactions that include zero fee get into the blockchain. A fee event would be a situation where blocks remain completely full for an extended period of time (for instance 7 days in Garzik's definition). In this situation, zero fee transactions would be rejected by the network and even some fees below a particular threshold would not be included into the blockchain, ever.

According to most in the community, this sort of 'fee event' would spell doom to the Bitcoin experiment and must be avoided at all cost. In his recent medium post, Mike Hearn, in concluding bitcoin has failed stated: "Why has Bitcoin failed? It has failed because the community has failed. What was meant to be a new, decentralised form of money that lacked 'systemically important institutions' and 'too big to fail' has become something even worse: a system completely controlled by just a handful of people. Worse still, the network is on the brink of technical collapse.". He went on to say, "The block chain is full. You may wonder how it is possible for what is essentially a series of files to be 'full'. The answer is that an entirely artificial capacity cap of one megabyte per block, put in place as a temporary kludge a long time ago, has not been removed and as a result the network’s capacity is now almost completely exhausted.". There are many others in the community that share this view point, that if Bitcoin blocks fill up, Bitcoin will be destroyed and not work properly.

In this post, I will attempt to explain my view (which may be considered heretical to some) that a fee event is a good thing because it indicates that Bitcoin has succeeded. Furthermore, I will argue that it something we should encourage to happen sooner, rather than later due to the fact that it is inevitable if Bitcoin is to succeed.

First, lets start with the numbers. In a recent block that was almost (973.8 kB), a total of 2081 transactions confirmed. As an approximation, lets say that each 1mb of block can contain approximately 2000 transactions. If blocks occur every 10 minutes, that implies a limit of around 3.33 transactions per second. This number may not be exact, but it is within an order of magnitude at least. According to recent work by Jonathan Toomim, a block size of 3mb is indicated to be the largest size that is safe. This number is based on his experiments on the testnet. So, that would imply a limit of about 10 transactions per second (3 X 3.33 ~= 10). With upcoming technology (like Xtreme Thinblocks and others), we might be able to increase this number, but even then we're ultimately limited by the fact that all transactions must be broadcast to all participants in the network. This results in an O(n^2) scaling which is generally considered not acceptable. Where does this put us in terms of other payment networks? According to VISA, their peak capacity is 47,000 tps. While their annual average appears to be around 2,000 - 4,000. According to 2011 numbers, 282 million people have VISA cards (or 4% of the world's population). Even for that 4%, they don't exclusively user their VISA cards for purchases. For Bitcoin, we want to do better than that, but as these numbers illustrate, even tens to hundreds of thousands of transactions would not satisfy the world's demand for payments. The block sizes required for these amounts are unfathomable today and, even if we experience 17.7% growth in bandwidth and other technologies (as indicated as the growth rate of bandwidth by Cisco referenced by Peter Wuille in BIP103), there is little hope for Bitcoin as is to scale to the levels required to support all the world's transactions any time soon. This is not even to get into settling stocks/bonds/and other securities on the blockchain.

This means, that without question, unless Bitcoin is a failure (or just a niche success), Bitcoin blocks will eventually fill up. Therefore, a fee event indicates success of the Bitcoin network because it means that demand outstrips supply. The real questions the community should be asking is: when do we want them to fill up? and what do we do when they do fill up?

Regarding the former question "when do we want them to fill up?", it seems that many in the community want the blocks to fill up as far into the future as possible. But, in reality, that is only delaying the inevitable. Blocks will fill up, and it may be hard to accept, but we need a plan for how to handle it. When thinking about this question, one thing to keep in mind is that currently the people involved in the Bitcoin community are generally very bright and generally in a better position to understand the situation and react than the broader population. Also, very few people are dependent on Bitcoin today. Therefore, we have an opportunity to build something that we can demonstrate works before mass adoption and work out the kinks so to speak. In my view, we should take advantage of this time we have and build a sustainable system. This would lead me towards the conclusion, that we should let the fee event happen now and deal with the consequences while we're still young. Once the block size limit is reached, the block size can be increased to lower fees as long as it doesn't bring us beyond the true capacity of the system.

That brings us the question of what do we do when the blocks fill up? Fortunately, even if we do nothing at all (or practically nothing), Bitcoin could most likely be used as a new global reserve currency and serve as a more fair, secure, and verifyable gold standard. See a blog entry from 2011, in which Nick Szabo indicated (in the comment section) that he thought that "Bitcoin may be destined to become a high-powered money rather than a day-to-day payment system for the masses.". While this isn't the most ambitious of goals for the Bitcoin community, it's clearly an improvement over the present day system. In that scenario, intermediaries like Coinbase and other Bitcoin banks would use Bitcoin to settle with each other once a day and allow instant, low cost, micropayments between users without including the transactions on the block chain. The Bitcoin banks would settle with each other periodically or possibly use payment channels to do so instantly. The main benefits over the existing system is that fractional reserve banking would be impossible because consumers would demand to see proof of reserves from their bitcoin banks. Proof of reserves could also serve to reduce the costs of auditing and make it reliable as well. The controlled (and known) supply of Bitcoin is superior to the properties of Gold because we know how much Bitcoin exists at any given time and don't have to worry about a big discovery, or something similar. We'd also have the benefits of being on something much similar to a gold standard.

But more ambitious projects like Lightning Network promise to scale Bitcoin enough to bring it to the masses with more reasonable block sizes which are bigger than today, but within the realm of possibility in the near future. I'm optimistic that Lightning Network and other things (some yet discovered) will allow us to scale as we all hope, but in either case, my hope is the community will accept this reality and start planning for a realistic future for Bitcoin and allow it to reach it's true potential.

What the heck is Bitcoin Core thinking?

Yesterday, Eric Voorhees made a post on Reddit titled "IMHO BTC price will be weak until Core demonstrates competency in social consensus.". While I don't necessarily totally agree with this, I do see his point that Core (or other people that share the view of Core) can improve their communications and help build social consensus.

This blog post will be my attempt to help explain the thinking of Core. While I'm not a Bitcoin Core developer myself, I have been following this space as a Bitcoin holder for quite a while and also have 15 years experience as a software developer and software development manager. I believe I have an understanding of what the Bitcoin Core developers are thinking and will attempt to convey that.

In my experience as a developer, very frequently a product manager will come to you and say, I want you to implement X. If X is something technical, the next question is: why do you want me to implement X? In some cases, implementing X is the right solution, but in other cases it's not the best way from a development standpoint to achieve the goal that the product manager is attempting to achieve. Throughout the years, one of the frequent statements that I make to product folks is: "Just give me the requirements, don't get into implementation details". The reason that this statement is frequently useful is because it allows the product manager to focus on what they actually want and avoid dictating how to do engineering work. Engineering work is the task of the developer, not the product manager and we are trained to do it better. It's also our obligation to explain why we intend to implement the requirements the way we intend to implement them.

I believe that the Bitcoin ecosystem is focusing too much on implementation details and not on product requirements. What users are saying is: "I want a bigger block size now!". The block size is an implementation detail, not a requirement. The requirement is actually that users want to continue to have low fees. Users are also worried that Bitcoin's throughput will not keep up with demand for transactions. These are valid requirements, but the requirement is not to increase the block size, that's the implementation detail. In addition to the requirement that transaction fees stay low, there is the other requirement that Bitcoin remain decentralized. That requirement is more important than low fees because without decentralization, Bitcoin is no real different than legacy currency systems and thus has no real value.

So, given these real requirements (low fees for an increasing user base, and decentralization) we now leave it up to the engineers to come up with the solutions to these problems. That's what the two scalability conferences last year were all about. So, the engineers got together and discussed it and the solution they came up with is multi-fold:

SegWit

Lightning Network

IBLTs and Weak Blocks

A block size increase after these things are implemented A full technical discussion of the Core roadmap can be found here.

So, lets break this down. What do each of these things do? First SegWit. SegWit is a method to separate (Segregate) out part of the Bitcoin transactions called the signature (Witness). What this does is allow us to discard some of the data (the signatures) in blocks that are only needed at the time of validation over time. The benefit of this is that we can scale Bitcoin without some of the downsides of increasing the block size. The net-net of this is that we can expect somewhere between 1.7x to 2x increase in the number of transactions to fit into blocks based on the current proposal. As mentioned in the roadmap, this change is expected to be available in April, 2016. Now, this change alone does nothing, because wallets need to upgrade their code to support this change. But, several wallets have stated that they intend to roll out this change concurrently with the release of Bitcoin core's code. There is some analysis on how much of a cost reduction will be generated by SegWit that one can view here. So, once again, when we focus on what customers want (fees to remain low), all that one needs to do is upgrade to a SegWit enabled wallet (which should be available in April) and you will get somewhere near 50% reduction in cost of transactions and we're talking about a few months out.

Next, we have Lightning Network. There is a lot of technical detail to Lightning Network, but to summarize what it does is that it allows for effectively the same level of security of the Bitcoin blockchain without actually submitting most of the transactions to the blockchain. With this method, even with the current 1mb limit on block size, lightning network could allow 50 million users to do an unlimited number of transactions for very low fees. Full detail here. As seen in the link, modest increases to the block size would multiply this number such that vast number of Bitcoin users could engage in unlimited number of transactions. One of the additional benefits of Lightning Network is that payments are irreversible instantly and we no longer have to wait for multiple confirmations once you have your coins in the Lightning Network.

Next we have IBLTs and Weak Blocks. IBLT is relatively new data structure that allows for somewhat lossy data transmission. It was initially proposed for use in the Bitcoin network by Gavin Andreesen here. The basic idea is that you can compress the data into this new data structure and reconstruct the data even in the presence of partial loss of data. This is useful in the Bitcoin network because that's exactly what we have a network where some (or all) transactions may not be propagated to all nodes. Thus an IBLT allows for blocks to be transmitted much more efficiently and potentially cuts bandwidth in half. Weak blocks are a method for allowing miners to pre-send the blocks that they are working on so that when a "strong" block is discovered, the network generally knows which strong block was won and it has propagated through the network faster. These two techniques will allow for a massive improvement in bandwidth consumption of Bitcoin and when they are ready, a hard fork to increase block size could be implemented.

So, what does all of this mean? With the combination of these changes, we are looking at a MASSIVE increase in capacity of the network over the next 1-2 years which will keep transactions cheap and maintain decentralization. As I pointed out, those are the customer requirements, NOT a block size increase. That's the implementation detail only. There are also other technologies that are not included in this summary (e.g. Schnorr signatures, possibly technologies/combinations of IBLT/Weak blocks/other tech). This vision is actually much more ambitious than even a large block increase like BIP101. Even if we were able to support much larger blocks we couldn't support all the worlds transactions. Some have argued that we could come close with BIP101 (which may not be technically feasible), but I think this view is limited to something like replacing the VISA network. While replacing the VISA network sounds ambitious, if you look at all the worlds transactions (including cash payments, stock trades, etc), the VISA network is actually relatively small. With the technologies discussed in this blog entry and the Bitcoin Core roadmap, we could see a much more ambitious goal and really replace the vast majority of all transfers of value in the world. That would make the world a better place and meet the requirements of the users.

BlockArrayChains

Background

-------------------------------------------------------------------------

One of the major problems with Bitcoin revolves around scaling. There have been numerous proposals with respect to scaling. Some proposals involve a layer on top of Bitcoin (e.g. Lightning Network, Thunder Network, StrawPay, etc). Others are actual changes to the Blockchain data structure itself (e.g. TreeChains, SideChains). In this blog, I will propose a new data structure which is an alternative to these proposals on a more fundamental level. I will refer to this new data structure as an BlockArrayChain.

Motivation for BlockArrayChain data structure

-------------------------------------------------------------------------

The Blockchain is basically a linked list of a block which includes a hash of a set of transactions with a nonce which generates a hash value below a specified difficulty. The reason for the controversy around increasing the block size is because there is a trade off between increasing the block size and the decentralization of the network. The BlockArrayChain is designed in such a way that instead of a single block at each node in the list there is an array of blocks at each link in the list. This way, each array consists n blocks of a set size (e.g. 1mb). I will explain in detail how the BlockArrayChain works later, but for now I will describe the benefits of such a data structure. The basic benefit is that we can maintain the current level of decentralization by keeping individual blocks at 1mb, but still allow the size of the Blockchain to grow much larger. It's important to note that the BlockArrayChain does not change storage requirements as each node would still need to store all blocks (including each block in an array of blocks), but as has been noted the main issue with increasing the block size centers around the bandwidth required to send blocks across the network and the CPU time required to validate large blocks and for them to propagate through the network. It is also important to note that the same pruning techniques used in pruning could be used with BlockArrayChains. Therefore, there are significant benefits to implementing BlockArrayChains.

Detailed description of BlockArrayChain

--------------------------------------------------------------------------

To simply things, think of a block in the current Bitcoin Blockchain as the following:

1.) A current hash

2.) A prev hash

3.) A Nonce

4.) A set of transactions (each of which includes a TXID which is a hash of the individual transaction)

A BlockArrayChain is similar however, instead of a single prev hash it includes all prev hashes from an array of blocks that preceded it. Note: to save space would could make it a hash of the list of prev hashes or something of the sort.

To summarize a BlockArrayChain would include the following data:

A set of blocks that include:

1.) A current hash

2.) A list (or hash of) of the previous hashes

3.) A Nonce

4.) Number of blocks targeted in the array

5.) A set of transactions (the TXIDs are partitioned across the array of blocks such that each block contains their appropriate proportion of transactions).

Note that miners may choose any number of blocks to include in the array. The current hash is adjusted downward based on the number of blocks included in the array. So the difficulty required for an array of 3 blocks is 1/3 that of a single block so it's equally beneficial to mine various numbers of blocks. It would rationally be set based on the number of transactions that are unconfirmed at any given time.

Any number of blocks may be chosen by miners in the new set of blocks. This brings up the possibility that miners may attempt to mine a different number of array elements at any given level. This is resolved exactly the same way that orphans are resolved in Bitcoin.

A rational miner would attempt to mine the fewest number of blocks that allow all transactions to fit in them. As soon as a single block is mined, it is rational for miners to continue mining that number of entries in the array until that full array is found (there is a case where very high transaction fees make it more attractive to drop the smaller array, but generally with today's large block rewards, that's not an important consideration. The system still works in that case as well). At this point, the miners would move on the the next array of blocks in the same manner. Since miners would work on different array elements, the orphan rate would be unchanged from the traditional Blockchain orphan rates at any given size.

See figure below for an outline of what this would look like graphically.

At level "A", there is only a single block, at "B", there are two blocks. At "C", there are three blocks and at D, back to two blocks.