IndiaFOSS 2024 - Live Streaming Post Mortem
On Ground Volunteers: Me (aryak), mangesh
Ideas we implemented this time
Live Streaming this year was a bit more complicated than last time, since we decided to take up the task of self-hosting our entire live stream infrastructure, rather than using YouTube or Peertube like we did for IndiaFOSS 3.0. We used this project called Open Streaming Platform.
We setup our OSP instance, https://stream.fossunited.org on a Hetzner VPS, scaled up to 8 dedicated cores, 32 GB RAM and 80 GB Storage for handling the conference load. For storage, we decided to use an S3 bucket to store all the large media files.
Additionally, a plan was put in place and bash script made to setup automated uploads of each talk as a clip to our streaming platform.
Another improvement we tried to implement this year was the decision to use fresh thinkpads instead of personal laptops to prevent issues such as the mid-stream system crash that happened in IndiaFOSS 3.0.
For those who weren’t able to access our service, we also planned on setting up an RTMP Restream to YouTube (feature that is supported by OSP which allows it to push the streams it gets to another rtmp service like youtube server side)
Finally, as per feedback regarding IndiaFOSS 3.0’s live stream from many attendees and even Rahul (CEO), we also worked on vastly revamping and improving the live streaming designs as compared to 3.0.
State of Footage and service
Due to audi 1 networking getting choked mid-way on day 1, and audio related issues, the footage for talks before lunch on day 1 in audi 1 are inaccessible. However, our videographer has all footage stored safely, and rest assured, the talks will be uploaded soon.
Large parts of the stream on our platform (done from the thinkpad) had faulty audio due to the mistake outlined in the next section.
You can find the specific recordings here:
- Audi 1: FOSS United Streaming - Channels - Videos
- Audi 2: FOSS United Streaming - Channels - Videos
- YouTube: https://www.youtube.com/@FOSSUnited/streams
The Good, Bad and Ugly
Good
- The streaming done from our side was pretty useful in showing people outside the audis what talk was currently happening inside
- Streams done by the videographer to youtube worked flawlessly
Bad
- Announcement of live stream was done on the event day, so pretty much no one watched the stream (generally not more than 1-2 consecutive viewers)
- Majority of setup was done last minute on day 0
- Audi 1 networking choked mid-way on day 1, causing some of the stream footage to be in a pretty bad shape
- Restream failed in last minute, though the videographer saved us by doing the stream from his laptop instead
Ugly
- Interchange of the audio inputs from the 2 capture cards and last minute changes on the thinkpad caused a lot of audio related issues in Audi 1
- Lack of monitoring of the stream did not help either. I take full responsibility for this mistake
- The S3 bucket was setup on day 0, and while changing the docker mount points to use S3 instead of local storage, the source and target were interchanged, causing us to almost run out of storage by the end of day 2 (thank god the 80 GB proved sufficient).
- The aformentioned auto-upload script was barely tested, and too many last minute changes resulted in it not working at all on the event day. It is also important to note that giving root ssh access to all the live stream machines for this upload script to work was a really stupid idea.
Ideas for the future
- Make this infra event-agnostic and make it suitable for use with cityFOSS conferences and meetups
- Suggestion to use live stream data to automate sending posts after every talk starts using a pre-defined template
- Having a separate network/line wherever possible for the live streaming so other users of the network cant hamper live streaming.
- Inform attendees and speakers regarding the live stream in advance
Conclusion
Even if there were a lot of problems, the stream ran very smoothly (whenever there was good network support) and the results are pretty good for this year.
Our infrastructure, scene files and scripts have all been dumped in a small repo I created, and I will work on making it event-agnostic and add proper documentation for it in the near future: GitHub - gi-yt/fossunited-streaming
We hope the stream will be even smoother in the coming years!