I've finally generated test material and tests to make a PR to fix text
segment generation for MPEG-2 TS streams with sparse teletext data,
making a big improvement compared to the original teletext support in
#1344.
## Problem
When packaging MPEG-TS input containing DVB-Teletext subtitles for
DASH/HLS output, two fundamental issues arise:
1. **Sparse input handling** - Teletext streams only contain data when
subtitles are displayed. During gaps (which can span multiple segments),
no PES packets arrive, leaving the text chunker with no timing
information to drive segment generation. This results in missing
segments or segments with incorrect timing.
2. **Misaligned segment boundaries** - Even when segments are generated,
the text segment timestamps and boundaries differ from video/audio
segments. This causes `<SegmentTimeline>` mismatches in the MPD,
playback issues on some players, and sometimes fewer text segments than
video segments.
## Solution
This PR introduces two complementary mechanisms:
### 1. Heartbeat mechanism (sparse input handling)
The `Mp2tMediaParser` now sends periodic "heartbeat" signals to text
streams:
- Video PTS timestamps are forwarded to all text PIDs as
`MediaHeartBeat` samples
- `EsParserTeletext` emits `TextHeartBeat` samples when PES packets
arrive without displayable content
- `TextChunker` uses these heartbeats to drive segment generation even
during gaps in subtitle content
- Ongoing cues that span segment boundaries are properly split and
continued
A new `heartbeat_shift` stream descriptor parameter (default: 2 seconds
at 90kHz) controls the timing offset between video PTS and text segment
generation, compensating for pipeline processing delays.
### 2. SegmentCoordinator (segment boundary alignment)
A new N-to-N media handler (`SegmentCoordinator`) ensures text segments
align precisely with video:
- Passes all streams through unchanged
- Replicates `SegmentInfo` from video/audio `ChunkingHandler` to
registered teletext streams
- `TextChunker` in "coordinator mode" uses received `SegmentInfo` events
to determine segment boundaries instead of calculating from text
timestamps
This guarantees identical segment timelines across all adaptation sets.
## Testing
- **Integration tests** in `packager_test.cc`:
- `TeletextSegmentAlignmentTest.VideoAndTextSegmentsAligned` - Verifies
segment count, start times, and durations match between video and text
-
`TeletextSegmentAlignmentTest.VideoAndTextSegmentsAlignedWithWrapAround`
- Same verification with PTS timestamps near the 33-bit wrap-around
point (~26.5 hours)
- **Test files** (synthetic teletext with known cue timings at 1.0s,
3.5s, 13.0s):
- `test_teletext_live.ts` - Normal PTS range
- `test_teletext_live_wrap.ts` - PTS near wrap-around boundary
- **Unit tests** for `SegmentCoordinator` and updated `TextChunker`
tests
## Documentation
- Extended `docs/source/tutorials/text.rst` with DVB-Teletext section
covering:
- Page numbering (3-digit cc_index format)
- Heartbeat mechanism explanation
- Segment alignment behavior
- `--ts_ttx_heartbeat_shift` parameter tuning
- Troubleshooting guide
- Added teletext processing pipeline diagram to `docs/source/design.rst`
## Future work
The heartbeat and `SegmentCoordinator` mechanisms would likely benefit
**DVB-SUB (bitmap subtitles)** as well (Issue #1477) , which faces
similar challenges with sparse subtitle data in MPEG-TS input and
segment alignment. The infrastructure is now in place to extend this
support.
## Example usage
```bash
packager \
--segment_duration 6 \
--mpd_output manifest.mpd \
'in=input.ts,stream=video,init_segment=video/init.mp4,segment_template=video/$Number$.m4s' \
'in=input.ts,stream=audio,init_segment=audio/init.mp4,segment_template=audio/$Number$.m4s' \
'in=input.ts,stream=text,cc_index=888,lang=en,init_segment=text/init.mp4,segment_template=text/$Number$.m4s,dash_only=1'
```
Fixes#1428Fixes#1401Fixes#1355Fixes#1430
Replaces #1181
* Add support for EBU Teletext input following Level 1.5 of the core
specification ETSI EN 300 706 V1.2.1 (2003-04).
* Add support for webvtt in MP4 segments output.
Closes#272
---------
Co-authored-by: Marcus Spangenberg <marcus.spangenberg@eyevinn.se>
# LL-DASH Support
These changes add support for LL-DASH streaming.
**NOTE:** LL-HLS support is still in progress, but it's coming. :)
## Testing
`./chunking_unittest --gtest_filter="ChunkingHandlerTest.LowLatencyDash"`
`./media_event_unittest --gtest_filter="MpdNotifyMuxerListenerTest.LowLatencyDash"`
`./mpd_unittest --gtest_filter="PeriodTest.LowLatencyDashMpdGetXml"`
`./mpd_unittest --gtest_filter="SimpleMpdNotifierTest.NotifyAvailabilityTimeOffset"`
`./mpd_unittest --gtest_filter="SimpleMpdNotifierTest.NotifySegmentDuration"`
`./mpd_unittest --gtest_filter="LowLatencySegmentTest.LowLatencySegmentTemplate"`
Note, packager_test must be run from the main project directory
`./out/Release/packager_test --gtest_filter="PackagerTest.LowLatencyDashEnabledAndUtcTimingNotSet"`
`./out/Release/packager_test --gtest_filter="PackagerTest.LowLatencyDashEnabledAndUtcTimingNotSet"`
Add dash_accessibilities stream descriptor, which is a semi-colon
separated list of accessibility_scheme_id_uri=value. It is optional.
Add dash_roles stream descriptor, which is a semi-colon separated
list of strings. It is optional.
Closes#565.
Change-Id: Idb1c20bb410fdd016db07e11fe507c102a3dd8ea
This allows UDP receive buffer size to be increased to avoid potential
packet loss result from receive buffer overrun.
Another related flag is --io_cache_size. buffer_size in UDP options
defines the UDP buffer size of the underlying system while
io_cache_size defines the size of the internal circular buffer managed
by Shaka Packager.
Closes#411
Change-Id: I57c843a88f13da546417dadc2a78df2bee0a00f3
- Also fixed documentation missing --hls_playlist_type LIVE for HLS
live examples.
- Also updated packager.cc to use RETURN_IF_ERROR macro for
consistency.
Fixes#347Fixes#403
Change-Id: Idbccd7137b873170cd54e2c780bd554d25031a0b
http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#option-lists
RST only accepts two forms for option argument:
- Begins with a letter and consists of [a-zA-Z0-9_].
- Begins with "<" and ends with ">"; any characters except angle
brackets are allowed internally.
Updated documents to obey the above rule.
Also cleaned up the documents to follow the syntaxes defined in
http://docopt.org/.
Change-Id: I06c6fa6db524325373053b26fc99169469664f01
Packager uses ThreadedIO to write media segments and manifest /
playlists. There was a possibility that media segments write being
delayed and scheduled after updating manifest / playlists.
This CL fixes the race condition.
Also added a note on how segments can be synced to cloud storage to
avoid the race condition during file sync.
Also added a live WebM test.
Fixes#386.
Change-Id: Icf9c38cdec715fa3dc2836eab1511131e129fe41
The number of preserved segments outside live window can be
configured using flag --preserved_segments_outside_live_window,
which is default to 50, i.e. 5 minutes for 6s segment.
Note that the segment removal will be disabled if it is set to 0.
Only HLS live playlist and DASH dynamic MPD are affected by this flag.
- Also add end to end tests.
Fixes#223.
Change-Id: I8a566efebe2f1552c7d9509ab017bade5a4a1c98
- Add --time_shift_buffer_depth to HLS doc.
- Fix a problem in HLS tutorial
Partially address #304, fixes#305
Change-Id: I60bc1eb659a30beb0a6ab73433f4cc99a7deab07
