1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
|
---
breadcrumbs: []
page_name: audio-video
title: Audio/Video
---
Everything you need to know about audio/video inside Chromium and Chromium OS!
### Whom To Contact
It's best to have discussions on chromium-dev@chromium.org or
media-dev@chromium.org for media specific matters.
We are component
[Internals>Media](https://bugs.chromium.org/p/chromium/issues/list?can=2&q=component%3AInternals%3EMedia)
on the Chromium bug tracker.
### Documentation
See
[media/README.md](https://chromium.googlesource.com/chromium/src/+/HEAD/media/README.md).
For historical reference, here's the original [design doc for HTML5
audio/video](/developers/design-documents/video).
### Codec and Container Support
Container formats
* MP4 (QuickTime/ MOV / MPEG4)
* Ogg
* WebM
* WAV
* HLS \[Only on Android and only single-origin manifests\]
Codec formats (Decode Only)
==Audio==
* FLAC
* MP3
* Opus
* PCM 8-bit unsigned integer
* PCM 16-bit signed integer little endian
* PCM 32-bit float little endian
* PCM μ-law
* Vorbis
* AAC \[Main, LC, HE profiles only, xHE-AAC on Android P+\] \[Google
Chrome only\]
* AMR-NB \[Google Chrome OS only\]
* AMR-WB \[Google Chrome OS only\]
* GSM \[Google Chrome OS Only\]
==Video==
* VP8
* VP9
* AV1 \[Only Chrome OS, Linux, macOS, and Windows at present\]
* Theora \[Except on Android variants\]
* H.264 \[Google Chrome only\]
* H.265 \[Google Chrome OS on Intel Gen 11-based Chromebooks for
protected content playback only\]
* MPEG-4 \[Google Chrome OS only\]
### Code Location
**Chromium**
media/ - Home to all things media!
media/audio - OS audio input/output abstractions
media/video/capture - OS camera input abstraction
media/video - software/hardware video decoder interfaces + implementations
third_party/ffmpeg - Chromium's copy of FFmpeg
third_party/libvpx - Chromium's copy of libvpx
**Blink**
third_party/blink/renderer/core/html/media/html_media_element.{cpp,h,idl} -
media element base class
third_party/blink/renderer/core/html/media/html_audio_element.{cpp,h,idl} -
audio element implementation
third_party/blink/renderer/core/html/media/html_video_element.{cpp,h,idl} -
video element implementation
**Particularly Interesting Bits**
media/base/mime_util.cc - defines canPlayType() behaviour and file extension
mapping
media/blink/buffered_data_source.{cc,h} - Chromium's main implementation of
DataSource for the media pipeline
media/blink/buffered_resource_loader.{cc,h} - Implements the sliding window
buffering strategy (see below)
third_party/blink/public/platform/web_media_player.h - Blink's media player
interface for providing HTML5 audio/video functionality
media/blink/webmediaplayer_impl.{cc,h} - Chromium's main implementation of
WebMediaPlayer
### How does everything get instantiated?
WebFrameClient::createMediaPlayer() is the Blink embedder API for creating a
WebMediaPlayer and passing it back to Blink. Every HTML5 audio/video element
will ask the embedder to create a WebMediaPlayer.
For Chromium this is handled in RenderFrameImpl.
### GN Flags
There are a few GN flags which can alter the behaviour of Chromium's HTML5
audio/video implementation.
ffmpeg_branding
Overrides which version of FFmpeg to use
Default: $(branding)
Values:
Chrome - includes additional proprietary codecs (MP3, etc..) for use with Google
Chrome
Chromium - builds default set of codecs
proprietary_codecs
Alters the list of codecs Chromium claims to support, which affects
<source> and canPlayType() behaviour
Default: 0(gyp)/false(gn)
Values:
0/false - <source> and canPlayType() assume the default set of codecs
1/true - <source> and canPlayType() assume they support additional
proprietary codecs
### How the %#$& does buffering work?
Chromium uses a combination of range requests and an in-memory sliding window to
buffer media. We have a low and high watermark that is used to determine when to
purposely stall the HTTP request and when to resume the HTTP request.
It's complicated, so here's a picture:
[<img alt="image"
src="/audio-video/ChromiumMediaBuffering.png">](/audio-video/ChromiumMediaBuffering.png)
|
