Note: There is some breaking changes here. If you manually worked with the Enum names here, then some of them have changed to better reflect the code points usage.
Generally speaking it should not affect service code.
I'm not happy with the approach used here to make portable installations of Devine, therefore for now I will remove the information relating to portable installations.
For aria2c I've simplified the operation by offloading most of the work for creating a cookie header by just re-doing what Python-requests does. This results in the exact same cookies Python-requests would have used in a requests.get() call or such. It supports multiple of the same-name cookies under different domains/paths based on the URI of the mock request.
It also looks for the "expected 2 but parsed 1" error which is likely an error while parsing the WVD version field. If this happens, it will inform the user to use `pywidevine migrate`.
This is mainly to lessen confusion on service name typo's or new users getting used to the CLI.
It also changed the Exceptions on the methods of Service from ClickException to a KeyError since they are intended to be used on the core codebase outside of the context of Click.
This used to be used even before devine was public, but it was constantly changed back and forth between an urljoin(), another form of urljoin (something custom or something I can't remember), and an if check + addition.
However, I can confirm that a simple if check will not work as the Base URI might not even be in the same relative root. The if checks have also been inconsistent with some checking if it starts with http(s)://, and some checking if it does not have the base URI at the start of the string.
This if check method does not work as well as an urljoin() has the potential to. It also fixes some services as some HLS playlists would have the m3u8 URL on a completely different root, subdomain, or even domain, causing it to completely break when trying to download segments.
We cannot actually do this check. The Content-Length value will be the size after being further encoded or compressed. While we can find out what it was compressed with via the Content-Encoding header, we cannot match the downloaded length with the Content-Length header as requests will automatically decompress/decode according to the Content-Encoding header.
On new installs, or where the `WVDs` folder is not made yet, then the shutil.move() assumes it's a file path and moves the `.wvd` file to the WVDs folder path, as a file. If the folder existed but was empty, this error wouldn't have occurred.
DASH and normal URL downloads now both decrypt one large single or merged file after all downloads are finished. This leaves a bit of a "pause" between progress bar movement which looks a bit odd. So mark the track as being in a Decrypting state.
Since DASH doesn't have the ability to change keys dynamically per-track (Representation), there's no need for the DASH downloader to decrypt segments as they are downloaded (like HLS).
This halves the amount of processes needing to be opened as well as the I/O usage. It may result in noticeably lower CPU usage. Since the IOPS is lowered, you may even see an increase in download speed if downloading to something like a meh HDD.
This also fixes decryption in some weird edge-cases where decrypting each segment individually resulted in timestamp anomalies causing shaka to fail.
Some Servers may not response with the Content-Length header, even if it's from segmented media. I.e., if it's a subtitle URL. The requests downloader required the header to be present as it downloads each URL, which is not possible.
Now it tries to get it if possible, and verifies the download size with the Content-Length value if it could be obtained.
HEAD requests were made to sum a total file size of the download operation. However, the downloader is may be used on URLs where the content is not segmented media. Therefore, the server may not support or respond with the Content-Length header which causes the requests downloader to crash before it even gets a chance to begin downloading.
Even still, this total size value isn't really necessary, and would cause possibly 100s of HEAD requests (in quick succession of each other) on segmented sources. It would also add up-front delay before it actually starts to download.
The setup I had for using asyncio.run with functools.partial didn't actually pan out. A full pass-through lambda is required.
I've also moved the mapped downloader variable to the root of the downloaders package.
Most people used --skip-dl just to license the DRM pre-v1.3.0. Which makes sense, --skip-dl is otherwise a pointless feature. I've fixed it so that --skip-dl worked like before, allowing license calls, while still supporting the new per-segment features post-v1.3.0.
Fixes#37
segments[0] is the first tuple, of two values. The URL and an optional byte range. So this accidentally passed the tuple rather than the URL within the tuple.
Fixes#54
The change to pymp4 may result in it complaining when validating parameters of the tenc box data, i.e., if the version byte is not 0 or 1, e.t.c. It shouldn't do a ConstError on tenc boxes anymore as the definition of the tenc box has been much improved in pymp4 v1.4.0.
pymp4 was updated to automatically do this during parsing and building of tenc boxes. Therefore it would instead fail parsing if the version is not 0 or 1.
It may look like I'm downgrading, but I'm not.
1.5.0 was an incorrect version bump based on TrueDread's fork. v1.4.0 is the next release after, even though it's a lower version. v1.4.0 should be used instead.
Some services accidentally (? I presume) mix up the `tenc` box's data with that of an `avc1` box or similar. This causes total failure and crashing. However, in these scenarios there's usually a 2nd box further down the stream that is not an error and will parse correctly. So just skip these errors and continue.
This got 'broken' after moving to my fork of pymp4 because my fork has commits by TrueDread that add support for the vttc, payl, and sttg boxes, therefore they no longer contain `data` fields but rather specifically parsed fields. I also no longer need to parse the data stream of vttc boxes, as they are already parsed as `children`.
It seems tenc boxes should only be 0 or 1, yet sometimes it can be above that. It seems some services accidentally use the `tenc` atom code over another. E.g., Netflix sometimes using `tenc` instead of `avc1`.
It's not very appealing, nor is it info the user realistically needs for every download. They could check `devine env info` to see where downloads go if they are unsure.
This happens because the input file has no actual useful data, likely just a bunch of headers and nothing of use. Therefore shaka skips and returns OK, yet didnt make any file at the decrypted path.
This fixes a crash when it tries to move the non-existent decrypted file to the input file location, causing the rest of the download to halt.
Negative size values are not allowed by the spec basically anywhere in the document. Some services seem to accidentally specify a negative value which puts pycaption on a fritz.
While it already has anti-duplicate checks, these checks did not take into account the `*` indicator, or `from x vault` e.t.c. This rework of the duplicate check ignores all messages.
This will improve efficiency and accuracy of getting appropriate DRM systems when downloading segments.
This can dramatically improve download speed from less than 50 kb/s to full speed if the HLS playlist used a lot of AES-128 EXT-X-KEYs. E.g., a unique key for each segment.
This was caused because the HLS.get_drm function took EVERY EXT-X-KEY, checked for supported systems, loaded them, and returned the supported objects. This meant it could load possibly 100s of AES-128 ClearKey objects (likely requiring URL downloads for the key URI) causing a huge delay before downloading each segment.
If you use --cdm-only, you will end up licensing multiple times if the PSSH has more than one Key ID. While this is checked for, with the KID check now being more lenient, it will end up continuing and licensing again.
However, even with the original check code this would have been pointless. If the first license did not return a content key for a KID, then the next license call with the exact same parameters wouldn't have either
For ex., if a service has the same PSSH or license call for 720p and 1080p video tracks, but it doesn't return a KID for the 1080p track, then the previous code would return an error, even though it has enough content key data to continue.
With this change it now only raises the error if the track's exact KID was not licensed. This adds support to prepare_drm for specifying the track's KID.
- DASH and HSL tracks must now explicitly provide the URL to download as the init segment. This is because the original code assumed the first segment, or first init segment was the one that the caller wants, which may not be the case (e.g., and ad is the first init segment).
It now supports explicitly providing a byte-range to download, as well as modifying the fallback content size of 20KB.
It now also checks if the server supports the HTTP Range header and uses it over the hacky request-streaming method. It also checks for the file size and uses that over the fallback size as long as it's not bigger than 100 KB.
Overall it's now more dynamic to specific use-cases, and more efficient in various ways.
These only happen if we intentionally cancel the process, or failed. However, this is something that is generally obvious given the args, and when cancelling a wall of these logs would appear.
rlaphoenix