Overview

概述

ExoPlayer is a media player built on top of the MediaExtractor and MediaCodec APIs released in Android 4.1 (API level 16). At the core of this library is the ExoPlayer class. This class maintains the player’s global state, but makes few assumptions about the nature of the media being played, such as how the media data is obtained, how it is buffered or its format. You inject this functionality through ExoPlayer’s prepare() method in the form of TrackRenderer objects.

ExoPlayer是建立在MediaExtractor and MediaCodec這兩組api之上的播放器。

這兩組api是在api 16 即Android 4.1發布的,所以ExoPlayer的支持的最小api是api 16，但是由于ExoPlayer的build.gradle的minSdkVersion為9，所以在api《16時，也可以使用ExoPlayer的部分功能。ExoPlayer庫的核心類是ExoPlayer類。該類維護了播放器的全局狀態，同時也。比如如何獲取媒體數據，如何緩沖以及是怎樣的編碼格式。

ExoPlayer provides default TrackRenderer implementations for audio and video, which make use of theMediaCodec and AudioTrack classes in the Android framework. Both renderers require a SampleSourceobject, from which they obtain individual media samples for playback. Figure 1 shows the high level object model for an ExoPlayer implementation configured to play audio and video using these components.

ExoPlayer基于MediaCodec and AudioTrack提供了默認的音視頻的TrackRenderer實現。所有的renderers都需要SampleSource對象，ExoPlayer從SampleSource獲得media samples 用于播放。圖1展示了ExoPlayer是如何配置組合這些組件用于播放音視頻的。

Figure 1. High level object model for an ExoPlayer configured to play audio and video using TrackRenderer objects

TrackRenderer

A TrackRenderer processes a component of media for playback, such as video, audio or text. The ExoPlayer class invokes methods on its TrackRenderer instances from a single playback thread, and by doing so causes each media component to be rendered as the global playback position is advanced. The ExoPlayer library provides MediaCodecVideoTrackRenderer as the default implementations rendering video andMediaCodecAudioTrackRenderer for audio. Both implementations make use of MediaCodec to decode individual media samples. They can handle all audio and video formats supported by a given Android device (see Supported Media Formats for details). The ExoPlayer library also provides an implementation for rendering text called TextTrackRenderer.

 TrackRenderer可以處理媒體文件中的音頻、視頻以及文本。ExoPlayer會在一個單獨的播放控制線程里調用TrackRenderer實例中的方法。這一方式使得每部分媒體組件都在播放時呈現出來。ExoPlayer庫默認提供了MediaCodecVideoTrackRenderer和MediaCodecAudioTrackRenderer 類來分別實現視頻和音頻的呈現。MediaCodecVideoTrackRenderer和MediaCodecAudioTrackRenderer 類都使用了MediaCodec 來對media samples解碼。他們能夠處理所有在 Supported Media Formats 列出的媒體格式。ExoPlayer還提供了TextTrackRenderer 來實現對文本的呈現。

The code example below outlines the main steps required to instantiate an ExoPlayer to play video and audio using the standard TrackRenderer implementations.

下面的代碼展示了實例化一個ExoPlayer對象并通過它播放音視頻的主要步驟。其中使用的是標準的TrackRenderer實現。

// 1. Instantiate the player.
player =ExoPlayer.Factory.newInstance(RENDERER_COUNT);
// 2. Construct renderers.
MediaCodecVideoTrackRenderer videoRenderer =…
MediaCodecAudioTrackRenderer audioRenderer =...
// 3. Inject the renderers through prepare.
player.prepare(videoRenderer, audioRenderer);
// 4. Pass the surface to the video renderer.
player.sendMessage(videoRenderer,MediaCodecVideoTrackRenderer.MSG_SET_SURFACE,
        surface);
// 5. Start playback.
player.setPlayWhenReady(true);
...
player.release();// Don’t forget to release when done!

For a complete example, see the SimplePlayerActivity in the ExoPlayer demo app, which correctly manages an ExoPlayer instance with respect to both the Activity and Surface lifecycles.

Figure 2. Object model for an implementation of ExoPlayer that renders media formats supported by Android usingFrameworkSampleSource

The following code example outlines how the video and audio renderers are constructed to load the video from a specified URI.

下面的示例展示了如何從特定url構建renderer。

FrameworkSampleSource sampleSource =newFrameworkSampleSource(
        activity, uri,null,2);
MediaCodecVideoTrackRenderer videoRenderer =newMediaCodecVideoTrackRenderer(
        sampleSource,null,true,MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT,0,
        mainHandler, playerActivity,50);
MediaCodecAudioTrackRenderer audioRenderer =newMediaCodecAudioTrackRenderer(
        sampleSource,null,true);

The ExoPlayer demo app provides a complete implementation of this code in DefaultRendererBuilder. TheSimplePlaybackActivity class uses it to play one of the videos available in the demo app. Note that in the example, video and audio are muxed, meaning they are streamed together from a single URI. TheFrameworkSampleSource instance provides video samples to the videoRenderer object and audio samples to the audioRenderer object as they are extracted from the media container format. It is also possible to play demuxed media, where video and audio are streamed separately from different URIs. This functionality can be achieved by having two FrameworkSampleSource instances instead of one.

ExoPlayer demo項目里面的DefaultRendererBuilder類中完整的包含了上面的代碼。SimplePlaybackActivity 類使用其播放了一段視頻。注意，在示例代碼中，音頻和視頻是混合在一起的，這意味著視頻流和音頻流都來自于同一個uri。當采樣數據從媒體容器中從提取出來時，FrameworkSampleSource 實例把視頻采樣輸出到 videoRenderer 對象中，同時把音頻采樣輸出到 audioRenderer 對象中。當然我們也可以播放音頻和視頻流分開的媒體，不過這需要我們建立兩個FrameworkSampleSource 實例。

Providing media for adaptive playback

ExoPlayer supports adaptive streaming, which allows the quality of the media data to be adjusted during playback based on the network conditions. DASH and SmoothStreaming are examples of adaptive streaming technologies. Both these approaches load media in small chunks (typically 2 to 10 seconds in duration). Whenever a chunk of media is requested, the client selects from a number of possible formats. For example, a client may select a high quality format if network conditions are good, or a low quality format if network conditions are bad. In both techniques, video and audio are streamed separately.

ExoPlayer支持自適應的數據流，也就是說，可以在播放時根據客戶端的網絡帶寬和CPU的執行能力的改變，隨時的調整視頻質量。DASH和SmoothStreaming是這類技術的典型代表。他們都是一次一小塊一小塊地加載數據的（一般來說大概會有2到10秒的延遲）。在獲取到小塊數據之后，客戶端可以選擇相應地格式來解析。例如，在網絡環境好的時候，客戶端可以選擇高質量的格式，而網絡環境不佳時，可以選擇低質量的格式。不過，這類技術里面，其視頻流和音頻流是相互獨立的。

ExoPlayer supports adaptive playback through use of the ChunkSampleSource class, which loads chunks of media data from which individual samples can be extracted. Each ChunkSampleSource requires aChunkSource object to be injected through its constructor, which is responsible for providing media chunks from which to load and read samples. The DashMp4ChunkSource and SmoothStreamingChunkSource classes provide DASH and SmoothStreaming playback using the FMP4 container format. The DashWebMChunkSourceclass uses the WebM container format to provide DASH playback.

ExoPlayer通過ChunkSampleSource 類來實現適應性播放功能。每個ChunkSampleSource 對象都需要在構造時注入一個ChunkSource 對象。我們通過ChunkSource 對象來加載采樣數據，并輸出成ChunkSampleSource 類需要的小塊媒體數據。DashMp4ChunkSource 類和SmoothStreamingChunkSource 類分別提供了DASH和SmoothStreaming協議下的解決方案，不過這兩個類支持的都是FMP4格式。DashWebMChunkSource類 則實現了DASH協議下使用WebM格式解析的解決方案。

All of the standard ChunkSource implementations require a FormatEvaluator and a DataSource to be injected through their constructors. The FormatEvaluator objects select from the available formats before each chunk is loaded. The DataSource objects are responsible for actually loading the data. Finally, theChunkSampleSources require a LoadControl object that controls the chunk buffering policy.

所有的標準ChunkSource 實現都需要在構造器中注入一個FormatEvaluator 對象 和一個 DataSource 對象。

The object model of an ExoPlayer configured for a DASH adaptive playback is shown in the diagram below. This example uses an HttpDataSource object to stream the media over the network. The video quality is varied at runtime using the adaptive implementation of FormatEvaluator, while audio is played at a fixed quality level。FormatEvaluator 對象會在每個數據塊被加載之前選擇可用的格式。  DataSource 對象負責實際加載數據。

最后，ChunkSampleSources 需要一個LoadControl 對象來控制數據塊的緩沖策略。

The object model of an ExoPlayer configured for a DASH adaptive playback is shown in the diagram below. This example uses an HttpDataSource object to stream the media over the network. The video quality is varied at runtime using the adaptive implementation of FormatEvaluator, while audio is played at a fixed quality level.

下圖展示了如何實現DASH 自適應播放。此例中通過HttpDataSource 對象從網絡獲取數據流。使用FormatEvaluator 時，音頻流的質量是固定的，而視頻流的質量則是在運行時不斷變化的。

Figure 3. Object model for a DASH adaptive playback using ExoPlayer

The following code example outlines how the video and audio renderers are constructed.

下面的代碼標出了音視頻的renders是如何被構造出來的。

Handler mainHandler = playerActivity.getMainHandler();
LoadControl loadControl =newDefaultLoadControl(
        newBufferPool(BUFFER_SEGMENT_SIZE));
BandwidthMeter bandwidthMeter =newBandwidthMeter();

// Build the video renderer.
DataSource videoDataSource =newHttpDataSource(userAgent,
        HttpDataSource.REJECT_PAYWALL_TYPES, bandwidthMeter);
ChunkSource videoChunkSource =newDashMp4ChunkSource(videoDataSource,
        newAdaptiveEvaluator(bandwidthMeter), videoRepresentations);
ChunkSampleSource videoSampleSource =newChunkSampleSource(videoChunkSource,
        loadControl, VIDEO_BUFFER_SEGMENTS * BUFFER_SEGMENT_SIZE,true);
MediaCodecVideoTrackRenderer videoRenderer =newMediaCodecVideoTrackRenderer(
        videoSampleSource,null,true,MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT,
        0, mainHandler, playerActivity,50);

// Build the audio renderer.
DataSource audioDataSource =newHttpDataSource(userAgent,
        HttpDataSource.REJECT_PAYWALL_TYPES, bandwidthMeter);
ChunkSource audioChunkSource =newDashMp4ChunkSource(audioDataSource,
        newFormatEvaluator.FixedEvaluator(), audioRepresentation);
SampleSource audioSampleSource =newChunkSampleSource(audioChunkSource,
        loadControl, AUDIO_BUFFER_SEGMENTS * BUFFER_SEGMENT_SIZE,true);
MediaCodecAudioTrackRenderer audioRenderer =newMediaCodecAudioTrackRenderer(
        audioSampleSource,null,true);

In this code, videoRepresentations and audioRepresentation are Representation objects, each of which describes one of the available media streams. In the DASH model, these streams are parsed from a media presentation description (MPD) file. The ExoPlayer library provides aMediaPresentationDescriptionParser class to obtain Representation objects from MPD files.

在這段代碼中，videoRepresentations 和 audioRepresentation 是Representation 對象。它們各自表示了一種可用的媒體流。在DASH模型中，這些數據流從MPD文件中轉換而來，這一轉換過程通過MediaPresentationDescriptionParser 實現。

Note: Building Representation objects from MPD files is not required. You can build Representation objects from other data sources if necessary.

注意：注意不一定要從MPD中構建Representation對象。如果必要的話，你也可以從其他數據源創建Representation對象。

The ExoPlayer demo app provides complete implementation of this code in DashVodRendererBuilder. TheSimplePlaybackActivity class uses this builder to construct renderers for playing DASH sample videos in the demo app. It asynchronously fetches a specified MPD file in order to construct the requiredRepresentation objects. For an equivalent SmoothStreaming example, see theSmoothStreamingRendererBuilder class in the demo app.

ExoPlayer demo項目在 DashVodRendererBuilder 類中完全實現了這段代碼。SimplePlaybackActivity類使用DashVodRendererBuilder 構造了renders用以播放DASH樣例視頻。在demo中，異步獲取了指定的MPD文件，來構建Representation 對象。同樣地，demo中使用SmoothStreamingRendererBuilder 來實現相同的功能。

int maxDecodableFrameSize =MediaCodecUtil.maxH264DecodableFrameSize();
Format format = representation.format;
if(format.width * format.height <= maxDecodableFrameSize){
  // The device can play this stream.
  videoRepresentations.add(representation);
}else{
  // The device isn't capable of playing this stream.
}

boolean isAdaptive =MediaCodecUtil.getDecoderInfo(MimeTypes.VIDEO_H264).adaptive;

@Override
publicvoid onVideoSizeChanged(int width,int height){
  surfaceView.setVideoWidthHeightRatio(height ==0?1:(float) width / height);
}

MediaCodecVideoTrackRenderer videoRenderer =newMediaCodecVideoTrackRenderer(
        videoSampleSource,null,true,MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT,
        0,mainHandler, playerActivity,50);

許多組件在發生異常時，也會發出通知，有些異常會導致播放失敗，有些不會。那些不會導致播放失敗的異常也很重要，很可能會影響應用的性能，所以我們也會需要記錄所有的異常信息，以便追蹤。

需要注意的是，ExoPlayer實例發送給高級監聽者的異常，通常都是導致播放失敗的異常。如果是高級監聽者中接受到得異常，我們最后將錯誤信息告訴用戶。而對于獨立組件的監聽者來說，我們只需要對錯誤信息進行記錄即可。

Sending messages to components

發送消息給組件

Some ExoPlayer components allow changes in configuration during playback. By convention, you make these changes by passing asynchronous messages through the ExoPlayer to the component. This approach ensures both thread safety and that the configuration change is executed in order with any other operations being performed on the player.

一些ExoPlayer組件允許在播放過程中改變配置。為了方便起見，可以使用發送異步消息的方式變更配置。這樣可以確保線程安全，同時也能保證配置的更改按順序執行。

The most common use of messaging is passing a target surface to MediaCodecVideoTrackRenderer:

最常見的用法就是更改視頻顯示的surface：

player.sendMessage(videoRenderer,MediaCodecVideoTrackRenderer.MSG_SET_SURFACE,
        surface);

Note that if the surface needs to be cleared because SurfaceHolder.Callback.surfaceDestroyed() has been invoked, then you must send this message using the blocking variant of sendMessage():

注意，如果SurfaceHolder.Callback.surfaceDestroyed() 被調用即surface被銷毀時，我們需要發送一個阻塞消息給EXOPlayer：

player.blockingSendMessage(videoRenderer,
        MediaCodecVideoTrackRenderer.MSG_SET_SURFACE,null);

You must use a blocking message because the contract of surfaceDestroyed() requires that the app does not attempt to access the surface after the method returns. The SimplePlayerActivity class in the demo app demonstrates how the surface should be set and cleared.

這么做的原因是因為，surface被銷毀時，需要確保MediaCodecVideoTrackRenderer 不會再將數據寫入已經釋放的surface。

在demo應用中，有相關代碼可以展示如何正確地設置和清理surface。

Customizing ExoPlayer

One of the main benefits of ExoPlayer over MediaPlayer is the ability to customize and extend the player to better suit the developer’s use case. The ExoPlayer library is designed specifically with this in mind, defining a number of abstract base classes and interfaces that make it possible for app developers to easily replace the default implementations provided by the library. Here are some use cases for building custom components:

使用ExoPlayer的一個主要的好處是ExoPlayer能夠更加容易的擴展和定制化。ExoPlayer定義了許多朝向基類以及接口。這時代開發者能夠很方便的使用自己實現來替換默認實現：

• TrackRenderer - You may want to implement a custom TrackRenderer to handle media types other than audio and video. The TextTrackRenderer class within the ExoPlayer library is an example of how to implement a custom renderer. You could use the approach it demonstrates to render custom overlays or annotations. Implementing this kind of functionality as a TrackRenderer makes it easy to keep the overlays or annotations in sync with the other media being played.
• TrackRenderer - 我們可能會想要定制TrackRenderer 來處理視頻和音頻之外的其他媒體。TextTrackRenderer 就是一個自定義TrackRenderer -的實例。
• SampleSource - If you need to support a container format not already handled by MediaExtractor or ExoPlayer, consider implementing a custom SampleSource class.
• SampleSource - 如果你想要支持一個在MediaExtractor 或者 ExoPlayer的默認支持范圍之外的容器格式。可以考慮實現自定義的SampleSource類。
• FormatEvaluator - The ExoPlayer library provides FormatEvaluator.Adaptive as a simple reference implementation that switches between different quality video formats based on the available bandwidth. App developers are encouraged to develop their own adaptive FormatEvaluator implementations, which can be designed to suit their use specific needs.
• FormatEvaluator - ExoPlayer library provides FormatEvaluator.Adaptive as
• DataSource - ExoPlayer’s upstream package already contains a number of DataSource implementations for different use cases, such as writing and reading to and from a persistent media cache. You may want to implement you own DataSource class to load data in another way, such as a custom protocol or HTTP stack for data input.
• DataSource - ExoPlayer的上級包名下已經包含了一系列DataSource 實現，比如從媒體緩存池讀寫數據等。
• 你可以實現你自己的DataSource 類，以別的方式加載數據，比如自定義的協議或者HTTP stack 。

Custom component guidelines自定義組件的一些守則

If a custom component needs to report events back to the app, we recommend that you do so using the same model as existing ExoPlayer components, where an event listener is passed together with a Handler to the constructor of the component.

如果你得自定義組件需要有事件交互，我們推薦你使用和已有的ExoPlayer組件相同的編程模型。把 Handler 和事件監聽者通過構造器傳入組件中去。

We recommended that custom components use the same model as existing ExoPlayer components to allow reconfiguration by the app during playback, as described in Sending messages to components. To do this, you should implement a ExoPlayerComponent and receive configuration changes in its handleMessage()method. Your app should pass configuration changes by calling ExoPlayer’s sendMessage() andblockingSendMessage() methods.

對于組件的設置更改，我們也推薦你同已有的模型保持一致。不過這樣的話，你需要實現ExoPlayerComponent ，在handleMessage()方法中接受配置的變更。按照這種模式實現的自定義組件，就可以和默認組件一樣，通過調用sendMessage() 和blockingSendMessage() 方法來變更配置了。

Digital Rights Management

數字版權管理

On Android 4.3 (API level 18) and higher, ExoPlayer supports Digital Rights Managment (DRM) protected playback. In order to play DRM protected content with ExoPlayer, your app must inject a DrmSessionManagerinto the MediaCodecVideoTrackRenderer and MediaCodecAudioTrackRenderer constructors. ADrmSessionManager object is responsible for providing the MediaCrypto object required for decryption, as well as ensuring that the required decryption keys are available to the underlying DRM module being used.

在安卓4.3及以上系統中，ExoPlayer可以支持DRM技術。但是為了播放drm內容，你需要將一個DrmSessionManager 注入到MediaCodecVideoTrackRenderer 和 MediaCodecAudioTrackRenderer 構造器中。DrmSessionManager 對象提供了一個用于解密的MediaCrypto 對象，當然也要確保解密的keys是有效的。

The ExoPlayer library provides a default implementation of DrmSessionManager, calledStreamingDrmSessionManager, which uses MediaDrm. The session manager supports any DRM scheme for which a modular DRM component exists on the device. All Android devices are required to support Widevine modular DRM (with L3 security, although many devices also support L1). Some devices may support additional schemes such as PlayReady.

ExoPlayer庫提供了一個默認的DrmSessionManager 實現，叫做StreamingDrmSessionManager。StreamingDrmSessionManager 有用到MediaDrm. StreamingDrmSessionManager 支持任意一種設備支持的DRM架構。基本上，所有的安卓設備都支持Widevine modular DRM。也有一些設備能支持其他架構。

The StreamingDrmSessionManager class requires a MediaDrmCallback to be injected into its constructor, which is responsible for actually making provisioning and key requests. You should implement this interface to make network requests to your license server and obtain the required keys. TheWidevineTestMediaDrmCallback class in the ExoPlayer demo app sends requests to a Widevine test server.

使用StreamingDrmSessionManager 類時，需要注入一個MediaDrmCallback 到構造器中。MediaDrmCallback 用于獲取秘鑰。你需要實現這個接口，并通過網絡連接你的授權服務器來獲取密鑰。demo項目中的WidevineTestMediaDrmCallback 類就實現了通過網絡連接 Widevine測試服務器以獲取密鑰。