Streaming Video With Kivy
FleetingAfter understanding how to use the camera with kivy on android, I though about streaming.
In general, in my projects, I only need a recent-ish frame to have some feedback whether the phone is correctly positioned, so using grab_frame should be enough. Yet, it got me curious, so let’s dig in a bit!
MediaRecorder deals natively with MP4 and 3GP. But it saves the index, so called “moov atom” when closing the record. Therefore, the file is no readable when being recorded.
Yet, there exists a tool called untrunc that tries to fix it. It needs a reference file that is correct and will use it to fix the other one.
In a complete file, we can see the moov atom being put at the end, after the data (mdat).
AtomicParsley ok.mp4 -T
Atom ftyp @ 0 of size: 24, ends @ 24
Atom mdat @ 24 of size: 4259552, ends @ 4259576
Atom moov @ 4259576 of size: 5961, ends @ 4265537
Atom mvhd @ 4259584 of size: 108, ends @ 4259692
Atom trak @ 4259692 of size: 5845, ends @ 4265537
Atom tkhd @ 4259700 of size: 92, ends @ 4259792
Atom mdia @ 4259792 of size: 5745, ends @ 4265537
Atom mdhd @ 4259800 of size: 32, ends @ 4259832
Atom hdlr @ 4259832 of size: 44, ends @ 4259876
Atom minf @ 4259876 of size: 5661, ends @ 4265537
Atom vmhd @ 4259884 of size: 20, ends @ 4259904
Atom dinf @ 4259904 of size: 36, ends @ 4259940
Atom dref @ 4259912 of size: 28, ends @ 4259940
Atom url @ 4259928 of size: 12, ends @ 4259940
Atom stbl @ 4259940 of size: 5597, ends @ 4265537
Atom stsd @ 4259948 of size: 201, ends @ 4260149
Atom mp4v @ 4259964 of size: 185, ends @ 4260149
Atom esds @ 4260050 of size: 83, ends @ 4260133
Atom pasp @ 4260133 of size: 16, ends @ 4260149 ~
Atom stts @ 4260149 of size: 32, ends @ 4260181
Atom stss @ 4260181 of size: 256, ends @ 4260437
Atom stsz @ 4260437 of size: 4804, ends @ 4265241
Atom stsc @ 4265241 of size: 52, ends @ 4265293
Atom stco @ 4265293 of size: 244, ends @ 4265537
~ denotes an unknown atom
------------------------------------------------------
Total size: 4265537 bytes; 23 atoms total.
Media data: 4259552 bytes; 5985 bytes all other atoms (0.140% atom overhead).
Total free atom space: 0 bytes; 0.000% waste.
------------------------------------------------------
AtomicParsley version: (utf8)
------------------------------------------------------
Therefore, in a file not finalized, you can find the mdat, but no moov
AtomicParsley timelapse.mp4 -T
Atom ftyp @ 0 of size: 24, ends @ 24
Atom mdat @ 24 of size: 1048576, ends @ 1048600
------------------------------------------------------
Total size: 1048600 bytes; 1 atoms total.
Media data: 1048576 bytes; 24 bytes all other atoms (0.002% atom overhead).
Total free atom space: 0 bytes; 0.000% waste.
------------------------------------------------------
AtomicParsley version: (utf8)
------------------------------------------------------
That’s why the file cannot be read. It misses the instructions how to read it.
Untrunc will copy the moov instructions from a sane file and adjust the durations to whatever is in mdat.
untrunc ok.mp4 timelapse.mp4
Info: version '' using ffmpeg '7.0.2' Lavc61.3.100
Info: reading ok.mp4
Info: parsing healthy moov atom ...
Info: reading mdat from truncated file ...
0%
Error: unable to find correct codec -> premature end (~3.606%)
try '-s' to skip unknown sequences
Info: Found 7 packets ( mp4v: 7 )
Info: Duration of mp4v: 350ms (350 ms)
Info: saving timelapse.mp4_fixed.mp4
We get a valid file, that additionally is faststart.
AtomicParsley timelapse.mp4_fixed.mp4 -T
Atom ftyp @ 0 of size: 24, ends @ 24
Atom moov @ 24 of size: 693, ends @ 717
Atom mvhd @ 32 of size: 108, ends @ 140
Atom trak @ 140 of size: 577, ends @ 717
Atom tkhd @ 148 of size: 92, ends @ 240
Atom mdia @ 240 of size: 477, ends @ 717
Atom mdhd @ 248 of size: 32, ends @ 280
Atom hdlr @ 280 of size: 44, ends @ 324
Atom minf @ 324 of size: 393, ends @ 717
Atom vmhd @ 332 of size: 20, ends @ 352
Atom dinf @ 352 of size: 36, ends @ 388
Atom dref @ 360 of size: 28, ends @ 388
Atom url @ 376 of size: 12, ends @ 388
Atom stbl @ 388 of size: 329, ends @ 717
Atom stsd @ 396 of size: 201, ends @ 597
Atom mp4v @ 412 of size: 185, ends @ 597
Atom esds @ 498 of size: 83, ends @ 581
Atom pasp @ 581 of size: 16, ends @ 597 ~
Atom stts @ 597 of size: 24, ends @ 621
Atom stsz @ 621 of size: 48, ends @ 669
Atom stsc @ 669 of size: 28, ends @ 697
Atom stco @ 697 of size: 20, ends @ 717
Atom mdat @ 717 of size: 37815, ends @ 38532
~ denotes an unknown atom
------------------------------------------------------
Total size: 38532 bytes; 22 atoms total.
Media data: 37815 bytes; 717 bytes all other atoms (1.861% atom overhead).
Total free atom space: 0 bytes; 0.000% waste.
------------------------------------------------------
AtomicParsley version: (utf8)
------------------------------------------------------
We can do something similar:
- read the moov from a valid file
- open the streamed mdat data and decode it using the appropriate decoder.