### You really don't want to get this, but keep reading to know what this is. [360p version @ 128GB](https://nyaa.iss.one/view/1096843) | [1080p version @ 42GB](https://nyaa.iss.one/view/1097157) --- What's this? --- * This is a 13m Matroska file which contains a video track, two audio tracks, and a slightly larger than usual SSA subtitle track. * But, it's not your common file. The video track is just black, and is used only to set a resolution. Both audio tracks are fine and remuxed from the original. * The subtitle track... is somewhat uncommon. It contains 16894 Dialogue entries. For the 1080p version, the raw subtitle file size is a bit more than 1TB. For the 360p version, it's a bit more manageable at 128GB. * If you are able to play the file, and display everything correctly, you will see the same output as from the original. What are these magic subtitle files? --- * They contains 16894 line entries, which correspond to every frame from the original file. * Every line is composed of several colored vector drawings in the form of cubes. Each one has the size of 1x1 pixels. * These lines are aligned with the top left corner, and contain internal newlines to create rows and columns. * Every single of these pixels is colored. It is a lossless transofmation from the pngs produced to embedding them, so you could recover the original easily. * TL;DR: every line contains one frame of video. In the subtitles. Why is it Hardcoded and Softsubbed at the same time? --- * Original subtitles were burned into the frames used to compose these subtitles. * Playback and display of this information is 100% softsubs, so it is technically hardcoded/hardsubbed and softsubbed at the same time. * We could have had the lines split, sure. But isn't this worse? What are these video formats / encoding settings? --- * Video is only used as a background and not seen. So, anything works. * We obviously had to use h264-8bit@420fps Hi444 with crf51 to encode black frames on the 1080p version. * For the 360p version, we picked lossless h265-8bit@69fps for real life experience. Why is the 360p version larger than the 1080p version? --- * 1080p Matroska file has the subtitle track compressed. 360p does not, as it was manageable. * 360p version is just a rescale of original frames for "easier" playback on players, as 1080p version is too heavy. * If you extract the tracks, you will be able to get the original files with their large sizes. * You might need a patch to do this, provided later. How does it look? --- * When rendered properly, surprisingly, it looks the same as the original. [See this comparison of the first 1080p frame rendered out](http://screenshotcomparison.com/comparison/124771) for both ASS and the original. * As I was not able to play the 1080p version besides rendering out some manually picked frames, the rest of these screenshots come from the 360p wich you can mostly play back. * Most tests were done with mpv. At native resolution, the output looks like the original. Note that everything here comes from the subtitle files. *    * If you try to scale the playback window, the rescale breaks apart and you get to see some pixel drawing boundary artifacts. *  * For VLC, rendering was incorrect and produced funny artifacts and scaling issues, even at original playback resolution. In-program video capture did not work. *  * Other players were tried, but results were bad. mpv is recommended to try and play back these files. * mpv does run out of memory eventually. *  These tests were run on a machine with an [email protected] and 16GB of RAM. If you are able to playback the 1080p version with anything, comment as well. Would be interesting finding ways to make players handle these well. | Player | 360p | 1080p | 360p notes | 1080p notes | | :--- | :---: | :---: | :--- | :--- | | PotPlayer x64 | __NO__ | - | Very stuttery, does not render subtitles, freezes after a while. | | | GOM Player | __NO__ | - | Very stuttery, does not render subtitles, freezes after a while. | | | ffplay x64 | __NO__ | - | Very stuttery, does not render subtitles, freezes after a while. | | | MPC-HC x64 | __BADLY__ | __BADLY__ | Very stuttery, does not render subtitles. | [First frame can be rendered after tinkering](https://nyaa.iss.one/view/1097157#com-46) | | CCCP x64 | __NO__ | - | No playback after 10m. | | | SMPlayer | __BADLY__ | __NO__ | Correct subtitle display. Extremely stuttery, continously repeats sound track. Requires manual invocation of command as GUI thinks backend crashed. About quarter speed, drops most frames. Looks like it will run out of memory. | | | mpv | __YES__ | __MAYBE__ | Correct subtitle display. About half speed, drops some frames. After several minutes of playback, it runs out of memory. | Took ~10m to open window. UI unresponsive, shoots to 4-6GB usage, back at 2GB. After 20m, memory usage dropped to 100-200MB then shoots back to 6GB. Single-core used 100% all this time. 13h later, it rendered one audio frame, no actual video seen yet. | | VLC x86 | __BADLY__ | __NO__ | Incorrect subtitle display (shows grid artifacts, uses wrong sizes so it does not cover full frame). Very stuttery, only shows one frame every 10 seconds or so of playback. | Hogs single core of CPU. Nothing happened after 20m | Why is this Red? --- * It's based off [[finFAGs] Pop Team Epic - 01 (1920x1080 WEB AAC) [A0176128].mkv](https://nyaa.iss.one/view/995122). * At least, it matches this: `Remux of another uploader's original release for hardsubbing and/or fixing purposes.` * Don't blame the original group or Nyaa mods for this. It's 100% my fault this got here. This sounds insane. How long did it take you and WHY!? --- * The processing/creation of the actual .ass files took around 20h. * mkvmerge was not able to mux the subtitles, as it ran out of memory. During testing, it used more than the 128GB allocated to the test VM. Thankfully, we produced some patches to mkvtoolnix to make this possible and benefit everyone. - mkvmerge parses and loads the entire .ass file into memory to sort the entries. - Originally, it saved all the timing information, and line in memory. - After [this patch](https://pastebin.com/raw/zkT0YACS), it has to parse the file to extract the timing information, and keep a pointer and length to this information. - When it is muxing after parsing, it will read these lines back from the file. - This let us finish the muxing and compress the 1080p file. - During this process, I contacted the Matroska / mkvtoolnix peeps on their IRC channel. Logs are provided below. * The muxing of the 1080p version took one day and 3/4, or exactly 2577 minutes and 8 seconds. * We also tested playback on several players for both versions. * As for why... I wanted to advance our knowledge and limits of ASS/SSA. Also no one was able to stop my insanity. * And hey the file that created this is called [encodeStraightIntoASS.php](https://pastebin.com/raw/g2mu0tSb) Are you ok? --- * Does this seem ok? * Do you think I would be ok after also playing with these files' CRCs? * I also played with torrent files and generated a 2.6GB single .torrent file, but decided against releasing that one. * If that wasn't enough, I tried embedding both the mkvtoolnix.diff and encodeStraightIntoASS.php into the .torrent, but sadly this torrent failed loading into some clients but worked in others. * __All in all: I made this so you don't have to do it.__ ### Excerpts from nyaa IRC channel * [Full related logs over here](https://zerobin.net/?f2110ec065ec5ed7#aNEdfgUX6BmavQ0lnCquCOaTQlR4JBF0JbnMpwOZ7p8=), if you want to laugh :) ``` <DataHoarder> 20% done, 190GB of .ASS <DataHoarder> mind you single episode <Squiggy> That's terribly retarded <DataHoarder> indeed <DataHoarder> but someone needs to show how bad it is <DataHoarder> I don't think this universe needs more than one of this thing <Minami> the universe doesn't need more than 0 ``` ``` <DataHoarder> basically: video + softsubs -> hardcode subs into video -> create pngs for frames -> convert those pngs into Dialogue lines with pixel drawings -> profit <DataHoarder> now this hardsubbed video is technically 100% softsubbed <DataHoarder> you also can have some bullshit encode parameters <DataHoarder> as video track is useless <Etzimal> sounds insane <Etzimal> i love it ``` ``` <DataHoarder> > Remakes which reencode video to XviD or worse are not allowed. <DataHoarder> is ASS worse than XviD? <LaserEyess> no ``` ``` <ReimuHakurei> wow <ReimuHakurei> https://nyaa.iss.one/view/961922 your thing is better than this imo > [FeelsBadMan] SukaSuka [Twitch Chatsubs][WEB 1080p] :: Nyaa ISS > Anime - English-translated | 6.9 GiB | Uploaded by DataHoarder on 2017-09-23 <DataHoarder> ReimuHakurei: check the uploader <ReimuHakurei> oh <ReimuHakurei> OH <ReimuHakurei> PFFT ``` ### DataHoarder goes to #matroska * Sadly post was too large already. [Read full logs of the discussion here](https://zerobin.net/?0bc3559990a006ab#/CPLf+ly1CN6SMRLge+LxicQIqGzxMgtEBM95LvceOw=) * For context, Mosu is the mkvtoolnix dev