Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add background fetch to the library #879

Open
joeyparrish opened this issue Jun 13, 2017 · 12 comments
Open

Add background fetch to the library #879

joeyparrish opened this issue Jun 13, 2017 · 12 comments
Assignees
@theodab
Labels
component: offline The issue involves the offline storage system of Shaka Player priority: P2 Smaller impact or easy workaround type: enhancement New feature or request
Milestone
Backlog

Comments

@joeyparrish
Copy link
Member

joeyparrish commented Jun 13, 2017 •
edited
Loading

Once we have a PWA (#876), we should look into the use of background sync and/or background fetch to fetch segments for offline storage. This also depends on Fetch support (#829).

With this, one could begin storing content for offline consumption, then close the app and let the segments be fetched and stored in the background, as connectivity is available.
@joeyparrish joeyparrish added the type: enhancement New feature or request label Jun 13, 2017
@joeyparrish joeyparrish added this to the v2.3.0 milestone Jun 13, 2017
@joeyparrish joeyparrish changed the title Add background sync support to the PWA demo Jun 13, 2017
@joeyparrish
Copy link
Member Author

Updated to include both background sync and background fetch as tools we may want to use for this, and closing #881 as a duplicate.
@joeyparrish joeyparrish mentioned this issue Jun 13, 2017
Closed
@joeyparrish joeyparrish self-assigned this Jul 3, 2017
@joeyparrish
Copy link
Member Author

joeyparrish commented Oct 2, 2017 •
edited
Loading

Issues implementing background fetch & sync so far:

  • BG fetch is behind a flag in Chrome 61, experimental web platform features must be enabled explicitly. Expected at this stage, but worth noting.
  • navigator.serviceWorker provides no consistent indication of service worker existence. Sometimes, controller is null even when a SW is registered. Sometimes, controller becomes non-null after reloading the page. This means that we will need an app-level configuration to instruct us to try BG fetch in the first place, since we can't detect the SW.
  • navigator.serviceWorker.ready never resolves on localhost over http. It does on appspot over https. This means we can't reliably grab a reference to the SW registration, even asynchronously. Saving the reference in the app level after initial registration allows me to work around this for development purposes, but this is going to complicate library APIs for BG fetch. If this is consistently the case, we will have to have applications pass in a reference to the SW registration before we can use BG fetch.
  • At least on desktop Chrome 61 on Linux, the BG fetch API does not result in a network request. Nothing shows up in the Network debug panel in Chrome.
  • At least on desktop Chrome 61 on Linux, the BG fetch API does not provide any way to access the fetch. (May be related to the point immediately above.) The BackgroundFetchRegistration object contains icons, tag, title, and totalDownloadSize. The spec'd activeFetches does not appear. There is no clear way to access the results of the fetch.

Given all of the above, I am moving this issue to the backlog for now.
@joeyparrish joeyparrish added the flag: seeking PR We are actively seeking PRs for this; we do not currently expect the core team will resolve this label Oct 2, 2017
@joeyparrish joeyparrish modified the milestones: v2.3.0, Backlog Oct 2, 2017
@joeyparrish
Copy link
Member Author

According to Chrome team, background fetch is not ready for production yet. Leaving this issue on the backlog until that changes.
@joeyparrish joeyparrish removed the flag: seeking PR We are actively seeking PRs for this; we do not currently expect the core team will resolve this label Oct 3, 2017
@joeyparrish joeyparrish modified the milestones: Backlog, v2.4.0 Dec 4, 2017
@joeyparrish joeyparrish mentioned this issue Jan 30, 2018
Closed
This was referenced Feb 21, 2018
Closed
Open
@joeyparrish joeyparrish modified the milestones: v2.4, v2.5 Mar 4, 2018
@joeyparrish joeyparrish mentioned this issue May 30, 2018
Closed
@joeyparrish joeyparrish removed their assignment Nov 21, 2018
@joeyparrish joeyparrish modified the milestones: v2.5, v2.6 Nov 27, 2018
@joeyparrish joeyparrish modified the milestones: v2.6, Backlog Feb 12, 2020
@OrenMe OrenMe mentioned this issue Aug 3, 2020
Closed
@jimmywarting
Copy link

It would have been grate if the Native File System could be used somehow.
@joeyparrish joeyparrish mentioned this issue Mar 19, 2021
Closed
@joeyparrish joeyparrish changed the title Add background fetch to the PWA demo Apr 13, 2021
@joeyparrish joeyparrish mentioned this issue Apr 13, 2021
Closed
@theodab theodab self-assigned this Jul 13, 2021
shaka-bot pushed a commit that referenced this issue Aug 11, 2021
@theodab
feat(offline): Changed store order of operations.
5215f53 
This changes the order that things happen when the offline storage
mechanism stores a manifest, so that the manifest is made and the
segments are downloaded in separate steps.
This is in preparation for adding background fetch support.

Issue #879

Change-Id: I4451db839b654f6134f06a58c240a9ca98d31a4e
shaka-bot pushed a commit that referenced this issue Aug 13, 2021
@theodab
feat(offline): Load init segments first for keys
db8ad31 
It is unlikely that we will be able to load DRM sessions inside
the service worker for BG fetch. However, sometimes we have to get
the DRM keys from the init segments.
This changes Storage.downloadSegments_ to download the init segments
first if it looks like they will contain needed init data to create
license requests.
This also fixes a typo that was preventing us from getting init data
from segments, and adds a test that would catch that issue.

Issue #879

Change-Id: Ide859ed0eb2d9208150787f14d915135df681d96
shaka-bot pushed a commit that referenced this issue Sep 7, 2021
@theodab
feat(offline): Make segment storage stateless.
aded252 
This refactors the storage mechanism so that the method that
attaches a segment to the manifest is be a stateless async method,
and no longer needs to be in the same session as the method that
stored the manifest.
This is the end of phase one of the work towards allowing Shaka
Player to use background fetch to store assets offline.
This change will allow a service worker to store the segments as
they are downloaded, without having to keep the Shaka Player
instance alive.

Issue #879

Change-Id: I6a3545c57bacaf7229fe8c32669e88c6cc4e4138
@avelad
Copy link
Collaborator

avelad commented Sep 8, 2021

@theodab I have seen that with the latest changes all the initialization segments will be downloaded, even of qualities that are not going to be downloaded, is it intentionally?
@theodab
Copy link
Collaborator

theodab commented Sep 9, 2021

@avelad What assets are you seeing that happen on?

I tried downloading Angel One (multicodec, multilingual, Widevine) on both nightly and v3.2.0, and they seem to be downloading the same set of segments.
137 network requests, 35 items stored in the segment DB, manifest size reported as 8224778 in the DB... it all matches up.
@avelad
Copy link
Collaborator

avelad commented Sep 9, 2021

@theodab I have recorded a video, but it is too big to attach here, I have mailed it to you.
@theodab
Copy link
Collaborator

theodab commented Sep 9, 2021

We load the init segments so that we can create the segment index for those variants; that asset uses SegmentBase, so we need to see the sidx box from the init segment. It's part of the initial process of loading the manifest, so it happens before we choose which video stream we want to store.

It's not a new behavior. For example, I loaded that same asset you showed in v3.1.3 and the same number of network requests are made in that version.

We have been working on a variety of performance enhancements recently (see, for example, #1936). I believe one of the planned enhancements is to lazy-load the segment index, which should fix this problem.
@joeyparrish joeyparrish added component: offline The issue involves the offline storage system of Shaka Player priority: P2 Smaller impact or easy workaround and removed PWA labels Sep 27, 2021
@joeyparrish
Copy link
Member Author

This may be blocked on changes in Chrome for now: https://bugs.chromium.org/p/chromium/issues/detail?id=1256838
@joeyparrish
Copy link
Member Author

Possibly also blocked by this: https://bugs.chromium.org/p/chromium/issues/detail?id=1255711
@avelad
Copy link
Collaborator

avelad commented Jan 17, 2022

@joeyparrish I have seen that of the two issues that block the implementation, one is closed and the other has a commit, but it is not closed. Do you know if this is already working in the latest versions of Chrome?
@joeyparrish
Copy link
Member Author

We are waiting for the changes to become on-by-default and in the stable branch of Chrome. AFAIK that has not happened yet.
@theodab theodab mentioned this issue Jan 18, 2023
Open
@theodab theodab mentioned this issue Feb 15, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: offline The issue involves the offline storage system of Shaka Player priority: P2 Smaller impact or easy workaround type: enhancement New feature or request
4 participants
@jimmywarting @joeyparrish @avelad @theodab