Unified BeginFrame scheduling for Chrome
Unified BeginFrame scheduling for Chrome
Status: http://crbug.com/401331 and http://crbug.com/416760
Original Author: simonhong@chromium.org
Second Revision Authors: simonhong@chromium.org tansell@chromium.org (mithro@mithis.com)
Reviewers: brianderson@chromium.org, skyostil@chromium.org, nduca@chromium.org
- Last updated: March 1, 2015
Motivation
We've learned from previous study, Improved vsync scheduling for Chrome on Android, about how explicit BeginFrame notification message from browser to renderer can improve frame rate with input handling. We need to adopt this mechanism to other platforms such as aura or mac for improving performance and unified implementation on all platforms.
Goal
This document will propose new BeginFrame scheduling to achieve below goals.
- Separate BeginFrame ipc handling from OutputSurface to new handler
○ https://codereview.chromium.org/619843002/ (done by @simonhong)
- Make BeginFrame start by browser compositor’s scheduler
○ add forward_begin_frames_to_children in SchedulerSettings
○ https://codereview.chromium.org/723713003/ (done by @simonhong)
○ Refactoring is in progress (ProxyBeginFrameSource)
(https://codereview.chromium.org/845393002/ by @simonhong)
- Send BeginFrame message from browser to renderer on all platform
○ set true to forward_begin_frames_to_children in SchedulerSettings on browser side cc scheduler
○ Aura based implementation is WIP (https://codereview.chromium.org/775143003/ by @simonhong)
- Remove CompositorVSyncManager
○ We don’t need vsync synchronization with renderer process on unified BeginFrame scheduling
○ (https://codereview.chromium.org/775143003/ by @simonhong)
- Feeds system’s VSync to browser compositor's scheduler
○ Routes WindowAndroid::OnVSync to the scheduler in android
○ Routes Mac’s DisplayLink to the scheduler in MacOSX
○ Aura already sends vsync to browser compositor’s scheduler
- Use external BeginFrameSource for Mac & Android for feeding vsync params
○ Set true to use_external_begin_frame_source in SchedulerSettings and implements new external BeginFrameSources
- Remove ViewMsg_UpdateVSyncParameters
○ vsync parameter is no longer used in renderer process
- Limited use of self-ticking on renderer-side
○ In normal situation, scheduler of renderer process on all platforms will rely on external BeginFrameSource
○ Use un-throttled mode when renderer is in background mode
○ Use un-throttled mode when renderer’s frame-rate should be adjusted
- Consider re-usability with DisplaySurface which will become a parent scheduler
External dependencies
- STP should be act as a cc::Scheduler’s client
○ https://codereview.chromium.org/638653003/ (Done by @weiliangc)
- Make Android CompositorImpl use scheduled SingleThreadProxy
○ https://codereview.chromium.org/344743002 (@simonhong is working on this)
Benefits from unified BeginFrame (@brianderson)
- Coordination with input events. Android coordinates the input for us today, but for other platforms we need to coordinate the input events manually with the unified BeginFrame.
- After unified BeginFrame, the Browser will know about all active renderers before the renderer swap, which will allow us to:
○ Implement latency feedbackWait for multiple active renderers to swap in the browser, rather than only picking up the first renderer to swap
○ betwr aeen renderend browser (i.e. SwapNack). If an active renderer misses the browser’s deadline, the browser can tell the renderer that it was too slow immediately
- We can extend the info inside the BeginFrame once it is on all platforms. For example, it might make sense to include SwapAck/Nack, input events, and future sync points as part of the BeginFrame to reduce the number of messages and thread hops we have per frame.
- Unifying the code base across platforms
Terms
Sometimes, VSync and BeginFrame are used interchangeably. Below is my understanding about both terms in chromium code base.
- VSync: notification from system or OS level when system or OS want to output new frame
- BeginFrame: message that requests to compositor scheduler to trigger new frame.
You can easily think with above terms like this - “When browser process gets VSync notification from system, it send BeginFrame message to renderers to trigger their new frame if they needs”
Overall diagram (originated from tim’s)
https://docs.google.com/a/chromium.org/drawings/d/1WEj-6A-8FmJNIMbd9hvkvxAuOOTwQvkSjbKR79YCt-c/edit
Proposed implementations
Below diagram is more specific version of above diagram. Browser side scheduler ‘s BeginFrame message will be delivered to LTH and then ui::Compositor.
ui::Compositor will have BeginFrame observer list and it will toss BeginFrame to all observers which are DelegatedFrameHost. Whenever renderer side scheduler needs BeginFrame and doesn’t needed, its corresponding DelegatedFramHost will be added as an observer or removed.
Synchronising vsync params
We need to synchronize system’s vsync params with browser side scheduler when platform doesn’t support frame start callback. In this section, I’ll talk about how to modify current vsync update mechanism. Between browser process and renderer process vsync synchronization will be removed.
Current implementation
- Aura: CompositorVSyncManager is used to spread new vsync params to browser and renderers.
- Mac: RWHVMac fetches vsync params from DisplayLinkMac and sends it to renderer.
- Android: vsync params is not used.
Proposed implementation
With our next generation BeginFrame scheduling, we don’t need to send vsync params to renderer anymore because browser will request BeginFrame explicitly to renderer.
On Aura, only browser scheduler needs to be updated with vsync params.
Browser-side cc::Scheduler will use SyntheticBeginFrameSource to get next BeginFrame because Windows and Linux doesn’t provide frame start callback. Also, we can remove CompositorVSyncManager which is owned by ui::Compositor because vsync synchronization doesn’t need anymore.[1] [2] Renderer process will only rely on BeginFrame message from browser process.
On MacOSX, we don’t need to update vsync params with browser’s Scheduler anymore. Instead, we can use CVDisplayLinkOutputCallback which is called whenever the display link wants to output a frame by registering vsync callback via CVDisplayLink. CVDisplayLink will invoke registered callback when it needed a new frame. This callback will feed next BeginFrame to browser’s scheduler with externalBFS.
On Android[3] [4] [5] , there is no change with regards to vsync update.
We will use this vsync from android platform and deliver it to compositor scheduler by using externalBFS like mac when android starts to use SingleThreadProxy + Scheduler.
VSync-Aligned input handling
IIUC, When SendBeginFrame() is called at RWHV, BufferedInput[6] [7] (jdduke@ wrote VSync-Aligned Bufferred Input) should be delivered to renderer first before send BeginFrame to renderer. So we can have more chance to handle these inputs in the next frame.
Implementation plans
- Enable unified BeginFrame on aura (https://codereview.chromium.org/775143003/)
- Implements new DisplayLinkMac to receive vsync callback and deliver vsync params to browser side scheduler (which will be act like externalBFS)
- Unify code base with android platform
Question?
1. Does we need to modify current android implementation[8] [9] ? (solved - yes we need!)
2. Does we need to consider this feature in single thread mode[10] [11] , too?
BeginFrame API
See cc/scheduler/begin_frame_source.h file.
BeginFrameObserver
BeginFrameObserver {
OnBeginFrame(BeginFrameArgs)
BeginFrameArgs LastUsedBeginFrameArgs()
}
OnBeginFrame
Core functionality of the API, tells you that input has been processed and that now is a good time to begin rendering.
Notes: BeginFrame calls *normally* occur just after a vsync interrupt when input processing has been finished and provide information about the time values of the vsync times. However, these values can be heavily modified or even plain made up (when no vsync signal is available or vsync throttling is turned off).
LastUsedBeginFrameArgs
LastUsedBeginFrameArgs exists because any non-trivial observer needs to store BeginFrameArgs for usage until it receives the next BeginFrameArgs. The observer is also the authoritative source on if the BeginFrameArgs is going to be used in some way.
This functionality enables;
- Dropped frames can be detected by checking if the value is updated.
- Throttling, multiplexing and any other frame sources which needs information about the last information sent;
○ Don’t also need store a copy of the args.
○ Correctly take into account dropped frames.
BeginFrameSource
BeginFrameSource {
bool NeedsBeginFrames()
SetNeedsBeginFrames(bool)
DidFinishFrame(size_t remaining_frames)
AddObserver(BeginFrameObserver);
RemoveObserver(BeginFrameObserver);
}
NeedsBeginFrames / SetNeedsBeginFrames
Provides a level triggered system for enabling / disabling the generation of OnBeginFrame messages. The signal is level triggered to reduce IPC overhead when the source is not local to a process.
DidFinishFrame (optional)
Provides async back pressure information into the source about how many frames are still in the process of being processed. Is useful if a source wants to keep only a single frame pending or other similar options. These messages might be delayed by IPC overhead, so should be used as informational only.
AddObserver / RemoveObserver
Normal observer interface, probably implemented with base::ObserverList.
ProxyBeginFrameSource
CompositorVSyncManager is essentially there because it receives information from 2 threads:
1- one "authoritative" frame rate from XRandR, on the UI thread
2- the latest rate + alignment from the GPU, on the compositor thread.
Yes, CompositorVSyncManager should be existed until it is changed to run on SingleThreadProxy.
Before that, ui::Compositor should maintain two options.
If it is running on ThreadProxy(CrOS), CompositorVSyncManager is used as now.
If it is running on SingleThreadProxy(aura and mac with delegated rendering), it will use new BeginFrame scheduling.
+sievers@chromium.org who may have some thoughts here as it relates to recent Android browser compositor changes.
I think it's either UpdateVSyncParameters() *or* BeginFrame(). For the render compositors, we use the latter and don't need to tell those compositors about vsync params.
For the browser compositor, we currently do our own scheduling, but there are patches in the works for switching this to SingleThreadProxy+cc-scheduler+BeginFrame-scheduling.
+sievers@chromium.org, Will android use cc-scheduler as a BeginFrame source when the enne's patch (makes SingleThreadProxy as a SchedulerClient) is landed? If so, we should also update vsync params with cc-scheduler.
+tdresser@chromium.org might be able to chime in here. A slight difference between Android and Aura is that Aura's gesture detector is more or less a singleton, whereas on Android (for content-targeted input) we have one detector per view.
As touch input buffering occurs *before* gesture detection in the pipeline, there may be some subtle differences in how BeginFrame serves as an input flush signal.
Aura has one gesture detector per aura::Window. (GestureRecognizerAura is a singleton, but it stores a mapping from aura::Windows to GestureProviderAura's)
What needs clarification here?
I'm probably not the best person to speak on the ordering of the BeginFrame message and input.
Thanks for the write-up! It would be great if we could minimize the differences between the Android implementation and other platforms as much as possible. +sievers@chromium.org recently did some major cleanup on the Android browser side logic, so unifying it should be much easier now.
Thanks for reading, Sami!
I added proposed diagram for android, too.
Yes. All platforms are moving to Browser compositors that use SingleThreadProxy.
In this case, it'll actually be nice since you don't need to make a thread hop to get from the Scheduler to the RWHV's BeginFrameManager.
+brianderson@chromium.org, When browser compositor uses single thread + cc-scheduler, how we can get BeginFrame message? Does SyntheticBeginFrameSource take that role, too on main thread?
Unified BeginFrame scheduling for Chrome的更多相关文章
- WebRTC | Failed to execute 'setLocalDescription' on 'RTCPeerConnection': Failed to parse SessionDescription. a=msid: Missing track ID in msid attribute.
1.问题回放 使用如下代码获取局域网IP报错 (代码来源:https://github.com/diafygi/webrtc-ips 日期:2019-02-16) Uncaught (in promi ...
- puppeteer(五)chrome启动参数列表API
List of Chromium Command Line Switches https://peter.sh/experiments/chromium-command-line-switches/ ...
- Unified shader model
https://en.wikipedia.org/wiki/Unified_shader_model In the field of 3D computer graphics, the Unified ...
- requestAnimationFrame在Chrome里的实现
requestAnimationFrame是HTML5游戏和动画必不可少的函数,相对于setTimeout或setInterval它有两个优势,一是它注册的回调函数与浏览器的渲染同步,不用担心Time ...
- 精读《Scheduling in React》
1. 引言 这次介绍的文章是 scheduling-in-react,简单来说就是 React 的调度系统,为了得到更顺滑的用户体验. 毕竟前端做到最后,都是体验优化,前端带给用户的价值核心就在于此. ...
- OnCommand® Unified Manager
OnCommand Unified Manager Solution Components The following components are downloaded and installe ...
- Threading and Tasks in Chrome
Threading and Tasks in Chrome Contents Overview Threads Tasks Prefer Sequences to Threads Posting a ...
- Chromium Graphics : GPU Accelerated Compositing in Chrome
GPU Accelerated Compositing in Chrome Tom Wiltzius, Vangelis Kokkevis & the Chrome Graphics team ...
- PatentTips - Scheduling compute kernel workgroups to heterogeneous processors based on historical processor execution times and utilizations
BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates generally to h ...
随机推荐
- Making ViewState More Secure
Unencrypted view state in ASP.NET 2.0 could leak sensitive information https://www.rapid7.com/db/vul ...
- OC冒泡排序算法
面试的时候很多公司会要求写一个冒泡排序算法,于是用OC写了一个,代码如下所示 需要注意的事项:oc数组只能存放oc对象,因此遍历数组输出的时候,记得通过NSString对象转换成intValue #i ...
- POJ 3048 线性筛法求素数
一个坑: 有组数据如下: 1 1 坑很深-- //By SiriusRen #include <cstdio> #define N 200000 using namespace std; ...
- 关于javascript 与iOS交互的简单使用
关于导入 JavaScriptCore 这个就不说了 js交互我们首先用到webView与webView的代理 基本上是很简单,,不过后台10分的坑 以下是代码 在webView的加载完成里面 fun ...
- Sql Server 2012数据库的安装【自己一点一点敲的】
Sql Server 2012数据库的安装 1.到微软官网上下载 下载链接为:https://www.microsoft.com/zh-cn/download/details.aspx?id=2906 ...
- Windows 10 秋季更新错误
uefi 启动丢失修复 bcdboot c:\windows /s y: /f uefi /l zh-cn 0x80240034 你尝试安装的内部版本是有签名的外部测试版.若要继续安装,请启用外部测试 ...
- APP信息获取接口
https://itunes.apple.com/lookup?id=APPID&callback=2 http://myapp.com/cgi-bin/mapp/mapp_info?type ...
- Install the high performance Nginx web server on Ubuntu
Look out Apache, there's a web server – Nginx (pronounced Engine X) – that means to dismantle you as ...
- 「JavaSE 重新出发」01. Java介绍
「白皮书」关键术语 简单性(C++--) 面向对象 分布式 健壮性 安全性 体系结构中立 可移植性 解释型 高性能 多线程 动态性 Java 发展历程 SUN公司--Stanford Universi ...
- php获取js里的参数
php获取js的值有如下方式: 1.php echo出js文件得到返回值,在gamemap.js文件中输出参数. echo '<script type="text/javascript ...