(2021-12-20) My Proposal for a Semi-Decentralized Video Hosting System

This the third time I am rewriting this stupid post so I'll get right into it. YouTube has a bunch of problems that make people seek out alternatives, but a lot of the alternative platforms or video hosting systems I've seen have one or both of these problems:

  1. They'll eventually run out of money because their business model cannot keep up with ballooning video hosting costs.
  2. They are hard to understand, not user-friendly, or have limited platform availability.

The video hosting system I am going to propose here may not fully satisfy point 2 and will have some downsides that possibly only the most hardcore users would put up with, but point 1 will absolutely not be a problem. Point 1 will be such a non-problem that this system requires no special investments or startup funding (or ICO scams) and can be put together and stay together forever using a shoestring budget and a lot of free time. Also, a lot of the key building blocks for this system I will propose are already in place or will likely be in place soon enough, so it's possible that it will eventually arise naturally through people who never read this blog post, which would be cool.

So, what is this video hosting system I am proposing? Here it is: A federated network of video sites that don't need to host the video files users upload because users actually "upload" videos by seeding webtorrents through the site instead of sending their video file directly to the hosting site's servers to be hosted. (The federated part is technically optional, you can have a standalone video site do this too.) What does this mean and what is needed to get this to the point where it is rock-solid software people can rely on?

In case you are unfamiliar with what torrenting is and the terminology around it, here's a quick rundown: Torrenting is a way for people to share files over the internet by simply sending it directly from one computer to another computer without a central server hosting the file itself (peer-to-peer file transfers essentially). And the file download process is accelerated when multiple people are torrenting a file because they share the downloaded chunks of that file between each other, and then also become hosters of that file when they download everything. This means that someone who is sharing a file from their computer doesn't need to worry about their non-commercial internet connection being completely overloaded by tons of people downloading at once. And the person who originally uploaded a file can go offline or just stop hosting it, while the file is still kept up by other people who downloaded the whole thing. Torrenting also has the benefit of relative anonymity for the people who are sharing and downloading files, which is why it is a very popular method of pirating things on the internet, to the point that a lot of people think that torrenting is only for piracy.

In the torrenting world, a file hosted by someone is called a "seed", a person who is "seeding" (uploading) a file is called a "seeder", and a person who is downloading a file is usually called a "leecher". There are two ways of finding people with files: The first and original way is through a "tracker", where people broadcast what files they have available to a central server, and people who want those files can find out who has a file they are looking for by asking that server. The second way and newer way is through a DHT, which is a way to broadcast and find files without a central server. There are ".torrent" files which don't have the file itself, but provide your torrent client the instructions for where to find the file and how to verify that the file you got from other people is what you wanted. Then there are "magnet links" which is a way for your torrent client to find files on the DHT when you are not using a tracker to download files. One more term that is useful to know is "seedbox" which is a dedicated torrenting server that is either owned by you or a seedbox provider you rent one from.

So, torrenting originated with BitTorrent, which turned 20 years old this year. BitTorrent has always relied on dedicated torrenting clients as it is a protocol built atop of UDP. Torrenting by using the BitTorrent protocol is not possible with modern web browsers as they do not support the customizeable UDP connections needed to make a BitTorrent client inside of a webpage. However, modern web browsers are fully capable of making peer-to-peer connections to other web browsers (and other programs) by using WebRTC. Because of this capability you can do BitTorrent-like file transfers through web browsers, just not using the regular BitTorrent protocol. This is exactly what the WebTorrent project set out to do.

WebTorrent was originally made back in 2013, and it essentially is The BitTorrent Protocol, just done using WebRTC instead of the UDP implementation of BitTorrent. This means that you can have a "torrent client" in your web browser and send files directly to one or more other people who either have a web browser or a dedicated torrent client that supports WebTorrent. One thing that is great about WebTorrent is that videos don't need to be downloaded completely before they can be viewed, they can be viewed on demand in your browser, just like on a normal video hosting site. If you go to the WebTorrent website, you can find a list of cool projects that use WebTorrent. One of these projects is PeerTube, which is a federated video hosting system that does use WebTorrent, but only to partially offload the video hosting burden onto viewers, not as a way for people to seed and view video files on their web browsers. PeerTube still hosts files on "central" servers, not on users' computers or seedboxes.

To repeat, my proposed idea in this post is a federated network of video sites similar to PeerTube, but where people seed the video files themselves and just use the video sites as a video player, a content discovery/recommendation system, and a feedback mechanism (through comments/likes/whatever). Doing this is not exactly possible right now for a key reason: a lack of good and well-maintained dedicated torrent clients that support WebTorrent. You see, torrenting is resource intensive and web browsers are inefficient for that task, and I assume a lot people don't really want to keep a web browser open to keep a their video files available on the internet. (Here is the first major flaw with my proposed system, someone has to be seeding a video file constantly to have a video be up, you can't just upload a video to a site and forget about it.) The official WebTorrent project does include a Node.js implementation of the protocol that can be used to make dedicated torrent clients. However, Node.js is only slightly more efficient than web browsers, as javascript is inefficient compared to compiled programming languages such as C++ or Rust.

The dedicated torrent client problem can be fixed by coding a torrent client that supports WebTorrent with a compiled language. The building blocks for this happening are already in place, they just need to actually be... built. The libtorrent library is a popular open source C++ backend that is used by popular torrent clients such as Deluge and qBittorrent. That library did get WebTorrent support coded for it, but it has not yet been released as part of its feature updates. When that does happen, we'll finally get torrent clients that support WebTorrent, and they'll be available to all of the users of those popular torrent clients. Libtorrent is also used by a lot of torrent software used on seedboxes, so needing to keep your computer on constantly to have your videos up will not be a problem. If you're dying to have a dedicated WebTorrent client available that is coded with a compiled language right now and are able to code it, there is a WebTorrent-enabled fork of libtorrent that was made by the same guy that helped add WebTorrent support to libtorrent.

When all that happens, making video sites that allow you seed video files instead of uploading them normally will be a lot easier. So what will these sites be like, and how will they function? From a regular viewer's perspective they should mostly look and function like a regular video hosting site such as YouTube or Dailymotion. From a video creator's perspective things will be a little bit different. They will need to host their video files either on their own computer, or their own seedbox, or rent a seedbox from a seedbox provider. Then they will need to send the .torrent file or a magnet link to their hosting site when their video(s) are ready. The video site hosts will need to partially download the file to get its metadata before it can be put up on the site, but they won't need host the file. Video sites could provide a built-in tracker to make things smoother for both viewers and creators. Video sites also could opt to have their own built-in seedbox for people who want to pay to have their video files hosted for them, and they could also use their built-in seedbox system to help seed popular videos on their site. Since video files aren't hosted by the site and can go down if nobody is seeding them, video sites will have to keep track of what files are available or not to improve their recommendations and user experience. No viewer would like to come to a site and click a recommended video and then have just not work because the creator shut off their computer that was seeding the video file for the night.

Though the need to have at least one person seeding a video file on their computer or seedbox for a video to be up is a problem, it can be very useful when there are a lot of people who want to intentionally have a video file stay up. If there's a popular creator using this video hosting system, they can ask their fans to download a WebTorrent-compatible torrent client or get a seedbox and seed the files to keep the files up and well-seeded so more people can come view them. Perhaps you could make some sort of protocol that makes it really easy for a fan to just put in a URL or a code into a torrent client and the client automatically hosts all the video files that need to be hosted. This kind of thing may create the need for this video hosting system to have its own ecosystem of dedicated clients that are specifically made for it.

Now let's talk about the potential problems with this system, and how they could doom this idea to obscurity. I've already mentioned the need to have the video files be seeded by someone a few times, but there are a few more things. One of these is browser compatibility. I've checked how WebTorrent works in as many browsers as I could, and one that didn't work well was Safari on iPhone. WebTorrent there did make peer-to-peer connections, but it didn't seem to load the video player. I'm not sure if this is a problem in how iPhone Safari works, or if this is something that can be fixed by updating the WebTorrent browser code, but it's something that may partially reduce the potential of this video hosting system as iPhone Safari is a very popular web browser for a lot of people. All the other modern web browsers I've checked seemed to work fine (even iPad Safari). Another problem that could affect this idea is seed ratios, essentially the ratio of people seeding files to people leeching files. If enough leechers overload the people seeding files, there will be a lot of video buffering, which will ruin the user experience for a lot of people. One more technical issue is that this system would put a lot of bandwidth burden on its users, which would affect people who want to watch videos on limited mobile internet plans or people who live in developing countries where bandwidth is expensive.

For non-technical issues, things like people deciding to upload copyright infringing content could lead to a lot of cease & desist letters and lawsuits thrown around, which could hurt this video hosting idea, but if that does happen it would mean that the idea is popular enough to catch the attention of big companies, which is sort of a nice problem to have from the perspective of making this idea as widespread as possible. There's also people deciding to make and upload extremist content, which can be slightly fixed by having popular video sites moderate videos, but since this system is based on decentralized technologies, it can never really go away and should be expected to be an unavoidable aspect of the system. There are other problems I've probably not thought about, but I think I've listed enough here. I'm not sure how big these problems would end up being overall and how they would affect this proposed system if someone tried making this, but hopefully it works out and we can get this cool idea working.

That's all I really have to say about my idea. Thank you for reading. What are your thoughts on this?

◀  Return To Blog Feed