Commit Graph
100 Commits
Author SHA1 Message Date
rlaphoenix fd64e6acf4 refactor(utilities): Remove get_binary_path, use binaries.find instead
The function now located at core/binaries should only be used by services to find a specific binary not listed in there already, or if the name of the binary needed by your service differs.
2024-04-24 05:10:34 +01:00
rlaphoenix 677fd9c56a feat(binaries): Move all binary definitions to core/binaries file
This simplifies and centralizes all definitions on where these binaries can be found to a singular reference, making it easier to modify, edit, and improve.
2024-04-24 05:07:25 +01:00
rlaphoenix 9768de8bf2 feat(env): List possible config path locations when not found 2024-04-19 19:28:15 +01:00
rlaphoenix 959b62222e fix(env): List all directories as table in info 2024-04-19 19:27:33 +01:00
rlaphoenix c101136d55 refactor(Config): Move possible config paths out of func to constant 2024-04-19 19:23:56 +01:00
rlaphoenix 4f1dfd7dd1 refactor(curl-impersonate): Update the default browser to chrome124 2024-04-18 09:50:17 +01:00
rlaphoenix c859465af2 refactor(curl-impersonate): Remove manual fix for curl proxy SSL
The new version of curl-cffi includes the proper fix for applying ca-bundles to proxy connections making this manual fix no longer required.
2024-04-18 09:49:35 +01:00
rlaphoenix d1ae361afc docs(changelog): Add v3.3.2 Changes 2024-04-16 06:07:00 +01:00
rlaphoenix a62dcff9ad docs(version): Bump to v3.3.2 2024-04-16 06:06:44 +01:00
rlaphoenix 920ce8375b build(poetry): Update dependencies 2024-04-16 06:06:05 +01:00
rlaphoenixandGitHub 3abb869d80 Merge pull request #100 from retouching/patch-1
Check if width and height is digit if it's an str
2024-04-16 05:36:59 +01:00
rlaphoenix cbcb7e31b0 docs(contributors): Add retouching to Contributor list 2024-04-16 05:35:57 +01:00
rlaphoenix a850a35f3e fix(Basic): Return None not Exception if no proxy configured 2024-04-16 05:27:17 +01:00
rlaphoenix 09e80feee5 fix(cfg): Use loaded config path instead of hardcoded default 2024-04-14 03:44:30 +01:00
rlaphoenix f521ced3fe refactor(env): Use -- to indicate no config found/loaded 2024-04-14 03:42:41 +01:00
rlaphoenix b4e28050ab fix(env): List used config path, otherwise the default path 2024-04-14 03:35:17 +01:00
rlaphoenix 646c35fc1b fix(Subtitle): Optionalise constructor args, add doc-string & checks
Some HLS playlists can have extremely limited information so to accommodate this we need to make the Subtitle track support having almost no information. This isn't ideal but it's really the only solution.
2024-04-14 03:26:35 +01:00
rlaphoenix 7fa0ff1fc0 refactor(Subtitle): Do not print "?"/"Unknown" values in str() 2024-04-14 03:25:22 +01:00
rlaphoenix 5c7c080a34 fix(HLS): Ensure playlist.stream_info.resolution exists before use 2024-04-14 03:15:11 +01:00
rlaphoenix 1db8944b09 fix(HLS): Ensure playlist.stream_info.codecs exists before use 2024-04-14 03:14:45 +01:00
rlaphoenix 43585a76cb fix(Audio): Optionalise constructor args, add doc-string & checks
Some HLS playlists can have extremely limited information so to accommodate this we need to make the Audio track support having almost no information. This isn't ideal but it's really the only solution.
2024-04-14 03:13:46 +01:00
rlaphoenix 8ca91efbc5 refactor(Audio): Do not print "?"/"Unknown" values in str() 2024-04-14 03:12:50 +01:00
rlaphoenix 57b042fa4b refactor(Audio): List lang after codec for consistency with other Tracks 2024-04-14 03:08:40 +01:00
rlaphoenix 642ad393b6 style: Move __...__ methods after constructors 2024-04-14 03:07:23 +01:00
rlaphoenix 23485bc820 refactor(Video): Return None if no m3u RANGE, not SDR 2024-04-14 02:40:16 +01:00
rlaphoenix 15d73be532 fix(Video): Optionalise constructor args, add doc-string & checks
Some HLS playlists can have extremely limited information so to accommodate this we need to make the Video track support having almost no information. This isn't ideal but it's really the only solution.
2024-04-14 02:36:55 +01:00
rlaphoenix 9ddd9ad474 refactor(Video): Do not print "?"/"Unknown" values in str() 2024-04-14 02:32:34 +01:00
rlaphoenix dae83b0bd5 fix(Video): Ensure track is supported in change_color_range() 2024-04-14 02:31:31 +01:00
rlaphoenix 20da213066 docs(changelog): Add v3.3.1 Changes 2024-04-05 12:31:59 +01:00
rlaphoenix 36222972ee docs(version): Bump to v3.3.1 2024-04-05 12:30:43 +01:00
rlaphoenix 6a25b09301 build(poetry): Update dependencies 2024-04-05 12:30:09 +01:00
rlaphoenixandGitHub b7ea94de29 Merge pull request #95 from knowhere01/master
Events not working as expected
2024-04-05 12:25:33 +01:00
rlaphoenix e92f8ed067 docs(contributors): Add knowhere01 to Contributor list 2024-04-05 12:24:25 +01:00
rlaphoenix 994ab152a4 fix(requests): Fix multithreaded downloads
For some reason moving the download speed calculation code from the requests() function to the download() function makes it actually multi-threaded instead of sequential downloads.
2024-04-05 00:37:44 +01:00
rlaphoenix 5d1b54b8fa fix(Chapter): Cast values to int prior to formatting
If left as float, then it parses as e.g., `7.0` instead of `07`. This leads to the timestamp format being completely off.
2024-04-03 23:22:30 +01:00
rlaphoenix 10285c3819 feat(dl): Add *new* --workers to set download threads/workers
The previously named --workers which is now --downloads specified how many tracks to download, not how many threads/workers are used per-download.

It defaults to nothing, which each downloader then has their own defaults. All current downloaders though currently default to `min(32, (os.cpu_count() or 1) + 4)`, which is also the default for `ThreadPoolExecutor` in general.

This also brings a side effect of changing DASH and HLS's forced max_workers of 16 to now a more appropriate default but more importantly actually configurable. You can set a default in your config under `dl.workers`.
2024-04-03 00:58:47 +01:00
rlaphoenix 0cf20f84a9 refactor(dl): Change --workers to --downloads 2024-04-02 23:34:45 +01:00
rlaphoenix fb5580882b docs(changelog): Add v3.3.0 Changes 2024-04-02 21:59:43 +01:00
rlaphoenix 6d18402807 docs(version): Bump to v3.3.0 2024-04-02 21:53:07 +01:00
rlaphoenix 1db2230892 build(poetry): Update dependencies 2024-04-02 21:52:22 +01:00
rlaphoenix c3d50cf12c ci(pre-commit): Update hooks 2024-04-02 21:51:31 +01:00
rlaphoenix 5a12cb33e2 refactor(Track): Move from OnXyz callables to Event observer
Fixes #85
2024-04-02 18:01:03 +01:00
rlaphoenix 226b609ff5 feat(Events): Add new global Event Observer API 2024-04-02 13:00:38 +01:00
rlaphoenix c194bb5b3a fix(Basic): Fix variable typo regression 2024-04-02 11:06:34 +01:00
rlaphoenix 3b3345964a fix(WVD): Add exists/empty checks to WVD folder dumps 2024-04-01 18:36:51 +01:00
rlaphoenix f99fad8e15 refactor(WVD): Seperate logs in loop for visual clarity 2024-04-01 18:35:56 +01:00
rlaphoenix f683be01d4 fix(WVD): Move log with path before Device load
This is so if the Device.load() call fails, we can know which file of the many files failed.
2024-04-01 18:35:00 +01:00
rlaphoenix 9f4c4584da fix(WVD): Move log out of loop to save performance 2024-04-01 18:34:03 +01:00
rlaphoenix 117a1188cd fix(WVD): Fix empty path to WVDs folder check
It seems a change in click made it so this is now just an empty tuple rather than a one-item tuple with Path("") resolved.
2024-04-01 18:33:02 +01:00
rlaphoenix a053423d23 refactor(WVD): Print error if path to parse doesn't exist 2024-04-01 18:29:56 +01:00
rlaphoenix 3659c81d6a fix(WVD): Ensure WVDs dir exists before moving WVD file
Fixes #83
2024-04-01 18:28:21 +01:00
rlaphoenix 491a0b3a5a feat(Basic): Allow proxy selection by index (one-indexed) 2024-04-01 17:50:08 +01:00
rlaphoenix b36befb296 feat(Basic): Allow single string URIs for countries 2024-04-01 17:47:51 +01:00
rlaphoenix 03b8945273 docs(config): Explain brief usage of proxy_provider data 2024-04-01 17:46:58 +01:00
rlaphoenix 6121cc0896 refactor(Basic): Improve proxy format checks 2024-04-01 17:38:04 +01:00
rlaphoenix bd8309e1d7 fix(Basic): Make query case-insensitive
Fixes #88
2024-04-01 17:32:52 +01:00
rlaphoenix f25d2419cf fix(curl-impersonate): Set Cert-Authority Bundle for HTTPS Proxies
For some reason curl-impersonate (curl_cffi project) does not set the certificate-authority bundle for proxies, which to be fair is for some reason seperated into two curl-options.

Doing this change as well as removing the https->http scheme enforcement on proxies, fixes HTTPS proxies on the curl-impersonate downloaders. I also simplified the seperate http and https proxy definitions to the `all` definition which was not originally supported but does seem to be supported as of v0.6.2.

I tested this on NordVPN proxies which are explicitly HTTPS-only and it does work.
2024-04-01 16:54:21 +01:00
rlaphoenix 45ccc129ce feat(dl): Try find SSAv4 fonts in System OS fonts folder
Currently only Windows is supported. Feel free to make a pull request to add Linux or mac OS support.
2024-03-27 06:01:57 +00:00
rlaphoenix eeab8a4f39 feat(dl): Automatically attach fonts used within SSAv4 subs
The fonts must be within the /devine/fonts folder. This folder location can be changed in the config. If a font is missing it will warn the user and continue.

Closes #82
2024-03-27 06:01:57 +00:00
rlaphoenix 057e4efb56 feat: Add support for MKV Attachments via Attachment class
You add these new Attachment objects to the Tracks object just like you would with Video, Audio, and Subtitle objects.
2024-03-27 06:01:56 +00:00
rlaphoenix a51e1b4f3c docs(changelog): Add v3.2.0 Changes 2024-03-25 03:54:14 +00:00
rlaphoenix 7715a3e844 docs(version): Bump to v3.2.0 2024-03-25 03:53:42 +00:00
rlaphoenix 16faa7dadf build(poetry): Update dependencies 2024-03-25 03:52:38 +00:00
rlaphoenix d9873dac25 fix(HLS): Delete video/audio segments after FFmpeg merge 2024-03-24 22:28:15 +00:00
rlaphoenix 774fec6d77 fix(HLS): Delete subtitle segments as they are merged 2024-03-24 22:27:32 +00:00
rlaphoenix e7294c95d1 fix(requests): Block until connection freed if too many connections 2024-03-13 17:15:13 +00:00
rlaphoenix 36b070f729 fix(requests): Manually compute default max_workers or pool size is None 2024-03-13 17:12:06 +00:00
rlaphoenix 458ad70fae fix(Video): Delete original file after using remove_eia_cc() 2024-03-12 11:08:15 +00:00
rlaphoenix 9fce56cc66 fix(Video): Delete original file after using change_color_range() 2024-03-12 11:07:40 +00:00
rlaphoenix 1bff87bd70 fix(requests): Set HTTP pool connections/maxsize to max workers
This allows requests to open and save/cache up to *max_workers* amount of TCP connections. In most situations it will still only save and re-use one TCP Connection since it always tries to re-use the connection if one is available.

However, in situations where downloads are from more than 10 Host/Port combinations (the default pool connections/maxsize) then this will improve download speeds.
2024-03-12 01:06:42 +00:00
rlaphoenix 5376e4c042 refactor(Service): Go back to the default pool_maxsize in Session
The pool_maxsize value here isn't actually doing much. It should have also been applied to pool_connections. What we realistically needed was just pool_block to prevent opening too much connections (causing a warning). The default pool_connections=10 and pool_maxsize=10 is fine. The downloader doesn't currently use this value.
2024-03-12 00:59:30 +00:00
rlaphoenix c77d521a42 refactor(Track): Default the track name to it's lang's script/territory
This allows you to override the whole track name instead of just prefixing before the script/territory. If you want no track name at all, you can set the track name to an empty string.

The script "Zzzz" (placeholder?) and territory "ZZ" (placeholder?) are not used. The script/territory values are only used if available and if necessary. I.e., fr-CA will use "Canada" but fr-FR will NOT use "France", it will be blank.
2024-03-10 15:19:39 +00:00
rlaphoenix f0b589c8a5 refactor(Track): Remove TERRITORY_MAP constant, trim SAR China manually
e.g., Hong Kong SAR China, Macao SAR China
2024-03-10 15:13:01 +00:00
rlaphoenix 4f79550301 fix(Track): Fix order of operation mistake in get_track_name 2024-03-09 19:56:41 +00:00
rlaphoenix 73d9bc4f94 fix(HLS): Remove save dir even if final merge wasn't needed 2024-03-09 19:44:40 +00:00
rlaphoenix 35501bdb9c fix(DASH): Fix merge regression from recent commit
An else tree was used in https://github.com/rlaphoenix/devine/commit/4d6c72ba307b366b3df72cc5243bf8b6a593de11 when it shouldn't have been.

Fixes #81
2024-03-09 17:52:50 +00:00
rlaphoenix 1d5d4fd347 fix(dl): Use click.command() instead of click.group() 2024-03-09 01:40:21 +00:00
rlaphoenix 4d6c72ba30 fix(DASH/HLS): Don't merge folders, skip final merge if only 1 segment 2024-03-09 01:37:55 +00:00
rlaphoenix 77e663ebee feat(search): New Search command, Service method, SearchResult Class 2024-03-08 21:32:55 +00:00
rlaphoenix 10a01b0b47 fix(Track): Compute Track ID from the this variable, not self 2024-03-08 19:22:33 +00:00
rlaphoenix 4c395edc53 fix(dl): Add single mux job if there's no video tracks
Fixes regression from v3.1.0 with --audio-only, --subs-only and --chapters-only.
2024-03-08 19:06:21 +00:00
rlaphoenix eeccdc37cf fix(MultipleChoice): Simplify super() call and value types
It was using the wrong instance, leaving the convert() method to seemingly default to str() for the returned chosen value types (or something, I don't really see why this works).
2024-03-08 17:09:20 +00:00
rlaphoenix 423ff289db feat(Track): Allow Track to choose downloader to use
The downloader property must be a Callable of the same signature as the aria2c, curl_impersonate, and requests downloader functions. You can pass it these functions by importing, or a custom function of a matching signature.

Note: It will still override the chosen downloader and use a fallback one in the case of using aria2c downloader but the download uses the HTTP Range header.

Closes #70
2024-03-08 16:48:44 +00:00
rlaphoenix ba801739fe fix(aria2c): Support aria2(c) 1.37.0 by handling upstream regression
From aria2c's changelog (2007-09-02):

```
Now *.aria2 contorol file is first saved to *.aria2__temp and if it is successful, then renamed to *.aria2.
This prevents *.aria2 file from being truncated or corrupted when file system becomes out of space.
```

It seems something went wrong in 1.37.0 resulting in these files sometimes not being renamed back to `.aria2` and then being left there for good. The fix for devine would be to simply detect `.aria2__temp` and delete them once all segments finish downloading. My only worry here is the root cause for why it has failed to rename. Did the download actually complete without error? According to aria2c's RPC, no errors occurred. There's no way to add support for Aria2(c) 1.37.0 without this sort of change as the files to seem to download correctly regardless of the file not being renamed and then deleted.

Fixes #71
2024-03-08 16:15:50 +00:00
rlaphoenix 79506dda75 chore(HLS): Remove commented-out code from get_supported_key()
This is code I forgot to remove while testing the HLS rework which released in v3.0.0.
2024-03-08 15:48:39 +00:00
rlaphoenix ccac55897c refactor(ClearKey): Only use User-Agent if none set in from_m3u_key 2024-03-08 15:45:52 +00:00
rlaphoenix e0aa0e37d3 feat(ClearKey): Pass session not proxy str in from_m3u_key method
This reduces the amount of connections being made by quite a bit for playlists that constantly change keys, or have new key data for every single segment (e.g., Pluto sometimes).

It also allows you to pass headers and cookies, while still also being able to supply a proxy.
2024-03-08 15:44:41 +00:00
rlaphoenix c974a41b6d fix(dl): Include chapters when muxing
This is a regression from the newer mux-job code that was brought in alongside the multiple `-r/--range` mux jobs feature in v3.1.0.

Fixes #79
2024-03-08 15:30:36 +00:00
rlaphoenix 2bbe033efb fix(Tracks): Improve constructor typing, add Chapter(s) to typing 2024-03-08 15:20:40 +00:00
rlaphoenix 5950a4d4fa docs(changelog): Add v3.1.0 Changes 2024-03-05 17:11:47 +00:00
rlaphoenix 8d44920120 docs(version): Bump to v3.1.0 2024-03-05 17:09:34 +00:00
rlaphoenix f8871c1ef0 docs(changelog): Add git-cliff configuration
Conventional Commit scopes don't seem entirely compatible with Keep a Changelog's sections/headers, so I have abandoned the Keep a Changelog sections/headers for custom ones that more accurately represent the commit's scope.
2024-03-05 17:08:26 +00:00
rlaphoenix f7f974529b build: Explicitly use marisa-trie==1.1.0 for Python 3.12 wheels
The current version of langcodes (v3.3.0) is quite old and doesn't have explicit support for Python 3.11+ yet. It does work on Python 3.12 but one of it's dependencies, marisa-trie==0.7.8, does not have wheels for Python 3.12.

By explicitly using the pre-release version of one of langcode's dependencies, language-data, which is what depends on marisa-trie, we can upgrade to marisa-trie==1.1.0 which does have a wheel for Python 3.12.
2024-03-05 16:31:25 +00:00
rlaphoenix 0201c41feb feat(dl): Support multiple -r/--range and mux ranges separately
Multiple -r/--range values can be used with multiple -q/--quality values.

Closes #63
2024-03-04 13:11:43 +00:00
rlaphoenix 6e8efc3f63 fix(HLS): Use filtered out segment key info
Also simplifies calculation of wanted segment range when decrypting. Instead of storing the starting segment index number with the encryption_data variable, we just grab the first segment that isn't already merged.

Fixes #77
2024-03-04 12:51:00 +00:00
rlaphoenix 499fc67ea0 feat(cli): Implement MultipleChoice click param based on Choice param
This can be used in-place to click.Choice() when you want to choose multiple values. Values must be separated by `,` character. This does mean the `,` character cannot be in the choice sequence.
2024-03-04 11:06:56 +00:00
rlaphoenix b7b88f66ce feat(dl): Change --vcodec default to None, use any codec 2024-03-04 10:41:07 +00:00
rlaphoenix 1adc551926 refactor(dl): Remove unused get_profiles() method 2024-03-04 09:31:15 +00:00
rlaphoenix 77976c7e74 feat(Subtitle): Convert from fTTML->TTML & fVTT->WebVTT post-download 2024-03-02 15:37:12 +00:00
rlaphoenix cae47017dc refactor: Move dl command's download_track() to Track.download() 2024-03-02 15:08:22 +00:00