Two diagrams side by side: a three-layer architecture where specialized app environments, a privacy solution and a validium feed into two rollups which settle to Layer 1, and a two-layer architecture where they settle to Layer 1 directly.
Vitalik explains a few different L3 possibilities. Diagram by Vitalik Buterin, from What kind of layer 3s make sense?

Here's my attempt to summarize Vitalik's recent post on what a Layer 3 might be, and where we might be heading.

We know Ethereum Layer 1 has limited block space and is expensive to use. In order to scale up the number of transactions, and reduce costs, layer 2 'rollups' perform actions in their own blockchains, which are then bundled and submitted to the layer 1 blockchain. The L2s utilize layer 1 as a settlement layer, leveraging Ethereum's superior security and decentralization. So what about layer 3? Can we just keep stacking bundlers on top of each other in order to keep multiplying our scalability? Not quite, says Vitalik. That may work for computational proofs, but for data compression we can't just compress and then recompress. It doesn't work that way.

Rather than generalized scaling, layer 3 may provide customized functionality, such as applications that need something other than the EVM (Ethereum Virtual Machine), or applications optimizing for non-scalability needs, such as privacy. Alternatively, layer 3 may offer scaling in a manner that is more specialized and distinct from what layer 2 rollups offer. Vitalik offers another possible use for layer 3. L2 rollups have a transaction time vs cost tradeoff. To rollup and commit to the layer 1 at a high frequency leads to higher costs per layer 2 transaction. Alternatively, the L2 might submit to the L1 on a less frequent basis, bundling more L2 transactions per submission — therefore accruing a lower financial cost per L2 transaction, but at the cost of a slower time to finalization. Vitalik proposes that a permissionless 'batch handler' could theoretically batch together submissions from multiple L2s, allowing an optimization for cost without compromising on time to finalization.

In his conclusion, Vitalik then discusses how when exploring different architectures one can get lost in questions of "what is a layer?" There are existing mechanisms for things like aggregation or privacy (like Tornado Cash) that we don't typically consider L2s. Vitalik gives his own criteria for what makes an L2 but states that the definitions are less important than considering what architectures make sense.

What comes next? Ethereum just completed the shift from Proof-of-Work to Proof-of-Stake, an upgrade I'm super excited about as I'm strongly opposed to 'proof-of-waste' blockchains. While 'the Merge' reduces Ethereum's energy consumption by over 99%, the upgrade doesn't address Ethereum's scalability and cost issues. So long as those issues persist we will continue to see activity, particularly innovation and experimentation, gravitate to cheaper chains such as Solana and Avalanche, or to L2s such as Arbitrum. Over time I would expect Ethereum's activity to become more boring as human readable transactions like NFT trades get replaced by submissions of rollups from L2s. The proliferation of L2s, L3s or whatever we want to call them, will allow for increased specialization and innovation.

Now Ethereum will at some future point complete upgrades that will solve its scalability and cost challenges: the so called Surge, Verge, Purge and Splurge. When this occurs much of the scalability and cost benefits of associated chains will be rendered somewhat moot, and perhaps activity and innovation migrates back to the Ethereum L1. Some decentralization maxis might get a centralization twitch at that thought. I'm generally a proponent of the idea that the more viable blockchains we have, the more decentralization. Let a thousand flowers bloom. However, in this case I think the ecosystem would benefit if scalability and cost were solved at Layer 1. That frees up L2s and other chains to focus on specialization rather than scaling or costs. As a software engineer I'm pleased that AWS essentially solved scaling for most software applications. The fact that engineers can deploy an auto-scaling group of servers at a click allows more time and mindshare to go into building great products. I'd be happy to see L1s step up and take care of any scaling and cost concerns as that may, counter-intuitively for some, give us the most robust ecosystem of L2s and L3s.

N.B. I hate bag bias in the crypto media space. For transparency's sake, the author owns <1 ETH, and a tiny amount of other L1 tokens (AVAX, FTM, SOL, ONE).