laudom/haruhi-dl
2
0
Fork 0
Code Issues 13 Pull requests Packages Projects Releases Wiki Activity
18560 commits 2 branches 27 tags 47 MiB
Commit graph

682 commits

Author SHA1 Message Date
Mattias Wadman 9e816eca8f [svt] Extract timestamp and thumbnail in more cases (#27130) 
Add timestamp, set to "valid from" which i think could been seen as publish time.
Add thumbnail in more cases, seems to was only done in the embedded data case for some reason.
Switch svtplay test url to an existing video and also one with no expire date.
Also add an additional thumbnail url test regex.
 2021-02-26 14:32:52 +01:00
Remita Amine 968583c56f [infoq] fix format extraction(closes #25984) 2021-02-26 14:28:13 +01:00
renalid f3c426a2ee [francetv] Update to fix thumbnail URL issue (#27120) 
Fix the thumbnail URL. The issue was here for many years, never fixed. It's done ! :-)

Example : https://www.france.tv/france-2/de-gaulle-l-eclat-et-le-secret/de-gaulle-l-eclat-et-le-secret-saison-1/2035247-solitude.html

failed thumbnail url generated : http://pluzz.francetv.fr/staticftv/ref_emissions/2020-11-02/EMI_1104da66f533cc7dc5d0d07a181a18c2e2fe1d81_20201014122553940.jpg

right thumbnail url fixed : https://sivideo.webservices.francetelevisions.fr/staticftv/ref_emissions/2020-11-02/EMI_1104da66f533cc7dc5d0d07a181a18c2e2fe1d81_20201014122553940.jpg
 2021-02-26 14:28:03 +01:00
Remita Amine 3ffb643844 [discoverynetworks] add support new TLC/DMAX URLs(closes #27100) 2021-02-26 14:19:12 +01:00
Remita Amine a732493292 [rai] fix protocol relative relinker URLs(closes #22766) 2021-02-26 14:19:07 +01:00
Remita Amine 493d279604 [rai] fix unavailable video format detection 2021-02-26 14:19:02 +01:00
Remita Amine a7bd83e154 [rai] improve extraction 2021-02-26 14:18:55 +01:00
Leonardo Taccari 9fd254036b [rai] Fix extraction for recent raiplay.it updates (#27077) 
- Remove first test of RaiPlayIE: it is no longer available
- Make RaiPlayIE extension-agnostic (passing possible `.json' URLs is now
  supported too)
- Adjust RaiPlayLiveIE to recent raiplay.it updates.  Passing it as
  `url_transparent' is no longer supported (there is no longer an accessible
  ContentItem)
- Adjust RaiPlayPlaylistIE to recent raiplay.it updates and instruct it about
  ContentSet-s.
- Update a RaiIE test and remove two tests that are no longer availables

Thanks to @remitamine for the review!
 2021-02-26 14:18:51 +01:00
Remita Amine ddc62043ed [viki] improve format extraction 2021-02-26 14:18:46 +01:00
beefchop 9adedd82f3 [viki] fix stream extraction from mpd (#27092) 
Co-authored-by: beefchop <beefchop@users.noreply.github.com>
 2021-02-26 14:18:39 +01:00
Remita Amine 339f127540 [amara] improve extraction 2021-02-26 14:16:30 +01:00
Joost Verdoorn 9a527679ed [Amara] Add new extractor (#20618) 
* [Amara] Add new extractor
 2021-02-26 14:16:03 +01:00
Remita Amine 514683921a [vimeo:album] fix extraction(closes #27079) 2021-02-26 14:13:40 +01:00
Remita Amine 46fce7272c [mtv] fix mgid extraction(closes #26841) 2021-02-26 14:13:32 +01:00
=?UTF-8?q?Sergey=20M=E2=80=A4?= 3a32ea072b [youporn] Fix upload date extraction and make comment count optional … 
…(closes #26986)
 2021-02-26 14:13:24 +01:00
=?UTF-8?q?Sergey=20M=E2=80=A4?= 1451f4f498 [arte] Rework extractors 
* Reimplement embed and playlist extractors to delegate to the single entrypoint artetv extractor
  Beware reluctant download archive extractor keys breakage.
* Improve embeds detection (closes #27057)
- Remove obsolete code
 2021-02-26 14:13:19 +01:00
=?UTF-8?q?Sergey=20M=E2=80=A4?= 93064492e9 [arte] Extract m3u8 formats (closes #27061) 2021-02-26 14:13:13 +01:00
Remita Amine f8fb198326 [mgtv] fix format extraction(closes #26415) 2021-02-26 14:13:08 +01:00
=?UTF-8?q?Sergey=20M=E2=80=A4?= e2b997d3bf [extractor/common] Output error for invalid URLs in _is_valid_url (re… 
…fs #21400, refs #24151, refs #25617, refs #25618, refs #25586, refs #26068, refs #27072)
 2021-02-26 14:13:00 +01:00
Remita Amine 9a4014d394 [francetv] improve info extraction 2021-02-26 14:12:48 +01:00
gdzx ff92752e7c [francetv] Add fallback video url extraction (#27047) 
Fallback on another API endpoint when no video formats are found.

Closes ytdl-org#22561
 2021-02-26 14:12:41 +01:00
=?UTF-8?q?Sergey=20M=E2=80=A4?= 9f47f2a04e [spiegel] Fix extraction (closes #24206, closes #24767) 
Code picked from PR #24767 since original repo is not available due to takedown.
 2021-02-26 14:12:29 +01:00
Remita Amine 14539655d5 [malltv] fix extraction(closes #27035) 2021-02-26 14:12:10 +01:00
Remita Amine 4826425743 [bandcamp] extract playlist_description(closes #22684) 2021-02-26 14:12:02 +01:00
Remita Amine 768e8bb238 [urplay] fix extraction(closes #26828) 2021-02-26 14:11:55 +01:00
Remita Amine ebc218c4c4 [lrt] fix extraction with empty tags(closes #20264) 2021-02-26 14:11:39 +01:00
=?UTF-8?q?Sergey=20M=E2=80=A4?= 883cf213dc [ndr:embed:base] Extract subtitles (closes #25447, closes #26106) 2021-02-26 14:11:26 +01:00
=?UTF-8?q?Sergey=20M=E2=80=A4?= ae004ab316 [servus] Add support for pm-wissen.com (closes #25869) 2021-02-26 14:11:12 +01:00
=?UTF-8?q?Sergey=20M=E2=80=A4?= 058b02f57f [servus] Fix extraction (closes #26872, closes #26967, closes #26983,… 
… closes #27000)
 2021-02-26 14:11:04 +01:00
=?UTF-8?q?Sergey=20M=E2=80=A4?= 51dd5a4cc5 [xtube] Fix extraction (closes #26996) 2021-02-26 14:10:56 +01:00
Remita Amine 2901a6439b [lrt] fix extraction 2021-02-26 14:10:38 +01:00
Remita Amine c62c95923a [condenast] fix extraction and extract subtitles 2021-02-26 14:10:03 +01:00
Remita Amine 44676b32c3 [bandcamp] fix extraction 2021-02-26 14:09:54 +01:00
Remita Amine d52a2bf577 [rai] fix RaiPlay extraction 2021-02-26 14:09:47 +01:00
Remita Amine 33c8322b1d [usanetwork] fix extraction 2021-02-26 14:02:17 +01:00
Remita Amine 4d26aa35af [nbc] fix NBCNews/Today/MSNBC extraction 2021-02-26 14:00:20 +01:00
Remita Amine b85fc0e982 [cnbc] fix extraction 2021-02-26 01:17:49 +01:00
Laura Liberda f42428e8e0 tvn24: next.js frontend extraction without playwright 
thanks to @ptrcnull
 2021-02-25 15:35:14 +01:00
Laura Liberda 07f5e2ae1c common: next.js data searcher 2021-02-25 15:22:17 +01:00
Laura Liberda 3ac7b35f1b PolskaPress extractor 2021-02-24 17:06:33 +01:00
Laura Liberda f91cd92ea0 [x-news] search for x-link in divs 2021-02-24 14:48:54 +01:00
Laura Liberda 8d30f19740 transistorfm extractors 
based on extractor by @asz: https://github.com/ytdl-org/youtube-dl/pull/28022
 2021-02-24 12:20:49 +01:00
Dominika eb0891a824 version 2021.02.23 2021-02-23 15:06:34 +01:00
Laura Liberda 84c08079bd [youtube] fix dynamic crypto 2021-02-23 14:01:39 +01:00
Dominika Liberda 13db87c0a2 + new crypto for YT 2021-02-22 20:31:44 +01:00
Dominika Liberda 85bb796e05 * jwplayer search meaningful error 2021-02-22 20:23:33 +01:00
Laura Liberda e35e90a263 [tvn24] fix age_limit on fakty 2021-02-12 13:16:04 +01:00
Laura Liberda 9bcb5f1b61 [tvn24] fix fakty extraction (closes #37) 2021-02-12 12:59:24 +01:00
Laura Liberda ac9192619a [pulsevideo] support age limit 2021-02-11 14:34:59 +01:00
Laura Liberda 4cfa7883a3 [clip.rs] fix extraction 2021-02-11 14:08:07 +01:00
Laura Liberda b55552ad1a [vod.pl] fix extraction 2021-02-11 13:18:57 +01:00
Laura Liberda 630a86c5e3 refactor from onet mvp to pulsevideo 2021-02-11 01:09:05 +01:00
Laura Liberda 46643f645d [pulsembed] fix 2021-02-10 22:06:47 +01:00
Laura Liberda 62008d3f8b pulsembed extractor 2021-02-10 01:55:47 +01:00
Laura Liberda d980ef4e35 [tvp:embed] fix DeprecationWarning 2021-02-10 01:50:08 +01:00
Laura Liberda bffba35446 [x-link] add iframe support 2021-02-10 01:39:10 +01:00
Laura Liberda 07f326f52d [tvp:embed] fix _extract_urls 2021-02-10 01:05:00 +01:00
Laura Liberda f2038499ef [OnetMVP] url extractor 2021-02-10 00:40:16 +01:00
Laura Liberda a60c736c44 [tvp:embed] new url scheme and url extractor 2021-02-10 00:39:27 +01:00
Laura Liberda edca3b8c96 simplify libsyn extraction 2021-02-10 00:38:40 +01:00
Laura Liberda b98dd103da [tiktok] user profile extractor 2021-02-05 16:37:22 +01:00
Laura Liberda a9cd876cf9 [lurker] domain update 2021-02-05 13:00:36 +01:00
Laura Liberda 87fad4b7eb x-link (x-news.pl embeds) extractor 2021-01-31 01:30:47 +01:00
Laura Liberda a3816f69be [generic] normalizing embeds part 2137 2021-01-31 01:26:39 +01:00
Laura Liberda b1c1d64de0 albicla extractor 2021-01-30 19:47:13 +01:00
Laura Liberda d2e522de09 wp.pl extractor 2021-01-25 15:45:55 +01:00
Laura Liberda 0b1dcd17b3 [tvn24] fix next.js frontend extraction on no cookies 2021-01-24 22:21:55 +01:00
Laura Liberda f39144fa0a [youtube] youtube.com/video/ url scheme 2021-01-24 20:14:01 +01:00
Laura Liberda d14745983d [peertube] reduce request amount if webpage downloaded 2021-01-24 06:13:30 +01:00
Dominika e65850dfd5 version 2021.01.24 2021-01-24 05:17:04 +01:00
Laura Liberda d08ac18c25 [tvn24] remove tvn24bis.pl references, remove GDPR consent cookies before opening page in browser 2021-01-24 04:56:39 +01:00
Laura Liberda 85e45ed607 remove phantomjs wrapper (closes #28) 2021-01-23 20:18:54 +01:00
Laura Liberda 95b061297c [pornhub] refactor scraping protection workaround from phantomjs to playwright 2021-01-23 20:15:27 +01:00
Laura Liberda 1285da6e17 [youtube] match /shorts/ urls 2021-01-23 15:42:50 +01:00
Laura Liberda 570cf794a9 [agora] wyborcza/wysokieobcasy/tokfm podcast fixes 2021-01-23 00:31:40 +01:00
Laura Liberda bed7b7f44c [crunchyroll] connect to https by default 2021-01-22 20:00:06 +01:00
Laura Liberda 339db8d1d2 [tvn24] support nextjs frontend (playwright required) and magazine 2021-01-22 19:59:00 +01:00
Laura Liberda 54ea13966e [youtube] minor playlist improvements 2021-01-19 18:45:18 +01:00
Laura Liberda c53f744097 playwright wrapper (#28) 2021-01-18 22:39:18 +01:00
Laura Liberda 365daad4f5 oko.press extractor 2021-01-18 03:52:28 +01:00
Laura Liberda abfbb7d014 _json_ld: podcasts objects 2021-01-18 03:52:19 +01:00
Laura Liberda 254f95b75f wyborcza extractors, merge agora extractors files 2021-01-17 08:41:00 +01:00
Laura Liberda d959fff39f lurker extractor 2021-01-17 06:08:30 +01:00
Dominika 2c354d15a0 version 2021.01.16 2021-01-16 02:38:11 +01:00
Laura Liberda 097c7bd4ba linkedin:post extractor 2021-01-11 17:58:24 +01:00
Laura Liberda 05620dab04 [weibo] dash formats 2021-01-10 23:36:29 +01:00
Laura Liberda eb1333e65b [vimeo:review] fix videos with video password 
https://github.com/ytdl-org/youtube-dl/issues/27591
 2021-01-10 10:42:15 +01:00
Laura Liberda 90291aa422 gtv.org extractor, I guess 2021-01-10 09:56:59 +01:00
Laura Liberda fe0f39da07 fix for youtube topic channels 2021-01-10 01:27:39 +01:00
Laura Liberda 90f556c478 [youtube] fix regular channels and playlists 2021-01-10 00:11:36 +01:00
Laura Liberda d51d7e3865 youtube music album extractor 2021-01-10 00:06:08 +01:00
Laura Liberda b044df8d86 LBRY extractor 
why did I do this?
 2021-01-09 04:31:11 +01:00
Laura Liberda 2acc7755fc [generic] embetty embeds 2021-01-09 02:11:47 +01:00
Laura Liberda 0abf03dbaf [heise] extract embetty embeds 2021-01-09 02:11:33 +01:00
Laura Liberda d72d60bf5e embetty extractor 2021-01-09 02:11:12 +01:00
Laura Liberda 2a772c54bb the guardian extractors 2021-01-08 21:01:36 +01:00
Laura Liberda cf9d87b0e6 [tvp:series] refactor to API 2021-01-08 12:41:55 +01:00
Laura Liberda 6e4b1019e4 basic tuba.fm support 2021-01-08 11:19:15 +01:00
Laura Liberda c758741d55 rmf extractors 2021-01-07 00:51:08 +01:00
Laura Liberda 0aa7cb240c _json_ld: handle multiple thumbnails 2021-01-07 00:49:10 +01:00
Laura Liberda 78d9e9046c [cda] refactor to mobile JSON API 2021-01-06 01:28:32 +01:00
Laura Liberda 1d601522cc a bit more embed searching normalization 2021-01-05 20:06:22 +01:00
Dominika Liberda 8f86520b7a version 2021.01.03 2021-01-03 23:30:31 +01:00
Dominika Liberda 5049bf7bea + Ipla Extractor (reverse engineered by @ptrcnull) 2021-01-03 23:16:16 +01:00
Laura Liberda 33c63089d9 tvnplayer:series extractor 2021-01-03 21:13:16 +01:00
Laura Liberda 837924dd1b fix tvnplayer on py2.7 2021-01-03 05:14:37 +01:00
Dominika Liberda 9bba7c1c5f + TVNPlayer Extractor (reverse engineered by @ptrcnull) 2021-01-03 03:52:08 +01:00
Laura Liberda 7ed8e0c502 tokfm podcast and audition extractor 2021-01-03 02:42:43 +01:00
Laura Liberda d915fe0b0c [generic] embed searching normalization 2/n 2021-01-01 07:05:16 +01:00
Laura Liberda c73049bc5b [youtube] alt_title w/ eng title, if it differs from the original one 2020-12-14 18:52:31 +01:00
Laura Liberda 49bf656179 [generic] simplify the embed searching a lot 2020-12-14 17:12:34 +01:00
Laura Liberda c9accf707d [funkwhale] fix TypeError on null album release date 2020-12-13 22:19:20 +01:00
Laura Liberda 048bac7c49 [funkwhale:channel] fix tests 2020-12-13 22:13:54 +01:00
Laura Liberda c3e1d87fcd [funkwhale] add webpage_url 2020-12-13 22:12:46 +01:00
Laura Liberda c40130632c [funkwhale] radio extractor 2020-12-13 21:57:22 +01:00
Laura Liberda 5592cda782 [tvp] abc.tvp.pl and general fixes 2020-12-13 03:12:31 +01:00
Laura Liberda 2d2abe34c8 [youtube] fix (dis)like count when it's 0 (#14) 2020-12-12 23:53:50 +01:00
Laura Liberda 572b04b7f1 [funkwhale] fix track duration 2020-12-12 23:11:58 +01:00
Laura Liberda bc1164719f [funkwhale] improve album data extraction 2020-12-12 23:05:16 +01:00
Laura Liberda e59434e242 [onet] remove extractors for dead services 2020-12-12 17:15:06 +01:00
Laura Liberda f646087983 [funkwhale] improve detection 2020-12-12 17:01:59 +01:00
Laura Liberda 4221c2ee68 funkwhale extractors 2020-12-12 06:12:40 +01:00
Dominika b2e1200c40 version 2020.12.11 2020-12-11 23:55:40 +01:00
Laura Liberda 32617e06b3 youtube:channel hotfix 2020-12-11 23:35:25 +01:00
Laura Liberda a12318ed7e shie: _match_id_and_host helper 2020-12-10 23:39:39 +01:00
Laura Liberda adea7807af mastodon extractor (#11) 2020-12-10 03:23:28 +01:00
Laura Liberda 0d8a0cefc1 fix suitable_selfhosted() on extractors with no regexes 2020-12-10 03:23:01 +01:00
Laura Liberda e33e398767 [generic] fix peertube embed tests 2020-12-10 01:46:57 +01:00
Laura Liberda b3c8623e4c [tvp:embed] refactored to TVPlayer2 API 2020-12-10 01:33:20 +01:00
Laura Liberda 889005bab3 selfhosted extractors, peertube extractor reworked (#10) 2020-12-09 21:52:30 +01:00
Laura Liberda 005b3fbedd eskago extractor (#20) 2020-12-08 00:16:49 +01:00
Laura Liberda dab3e41041 [atttechchannel] fix working, drop flash references 2020-12-07 04:37:02 +01:00
Laura Liberda a6102b5483 eurozet player extractors (#16) 2020-12-07 03:48:12 +01:00
Laura Liberda 24a54d5d52 eurozet article video extractor (#16) 2020-12-06 20:52:26 +01:00
Laura Liberda 4453792f0a [onet] amp urls 2020-12-06 04:07:00 +01:00
Laura Liberda 86629d8574 [onet] libsyn podcasts support, fixed tests 2020-12-06 03:51:23 +01:00
Laura Liberda 9b6ec60622 [onnetwork:frame] simplify code a lot 2020-12-06 01:17:44 +01:00
Laura Liberda 91d6c6dbd4 [polskieradio] livestream player extractor 2020-12-06 01:14:27 +01:00
Laura Liberda 6a4da9addf [polskieradio] new player_data thing 2020-12-05 20:01:55 +01:00
Laura Liberda ef114c2560 [polskieradio] support for polskieradio24.pl 2020-12-05 17:58:31 +01:00
Laura Liberda 3a38277a95 [cda] more fail-safe url replaces 2020-12-05 17:29:03 +01:00
Laura Liberda 33c19049af [cda] fix adult pages 2020-12-05 17:14:08 +01:00
Laura Liberda a11b8a8e5f [tvp] regional (client-side rendered) pages 2020-12-05 05:18:19 +01:00
Laura Liberda 07fa0508f9 [youtube] remove more useless tests 2020-12-05 01:09:29 +01:00
Laura Liberda 043bec4c9b [tvp] tvp stream support 2020-12-05 01:09:06 +01:00
Laura Liberda 10bb46f940 [tvp] polandin.com support 2020-12-04 19:37:08 +01:00
Dominika b3383de0a5 version 2020.11.27 2020-11-27 14:28:08 +01:00
Laura Liberda 40638606b9 [youtube] history, subscriptions 2020-11-24 21:48:21 +01:00
Laura Liberda 75c1755cc1 [youtube] liked, watch later support (#2) 2020-11-24 16:58:50 +01:00
Laura Liberda a3cba131b3 openfm extractor (closes #8) 2020-11-22 05:44:40 +01:00
Laura Liberda abdc94a6a5 ninateka extractor (closes #7) 2020-11-22 05:05:51 +01:00
Laura Liberda 7671ce8f00 [youtube] fix yet another UnboundLocalError 2020-11-21 00:56:20 +01:00
Dominika 486463ba53 version 2020.11.20 2020-11-20 13:36:08 +01:00
Laura Liberda c08b658033 [wykop] remove debug logs 😳 2020-11-18 17:51:46 +01:00
Laura Liberda 4eaf67ad2e [youtube] fix UnboundLocalError 2020-11-17 17:03:15 +01:00
Dominika a661491bcf removed more SWF references from youtube extractor 2020-11-17 13:56:32 +01:00
Laura c55393ce44 [youtube] fix channel/search on videos with no views 2020-11-16 02:31:20 +01:00
selfisekai 80a5d8d55e [youtube] fix channels with hyphen inside id 2020-11-16 02:31:20 +01:00
Dominika 98e6a95bd5 version 2020.11.16 2020-11-16 01:31:37 +01:00
selfisekai 6f876fba51 [youtube] fixed some download tests 2020-11-15 23:08:28 +01:00
selfisekai 158d4e9088 [youtube] search info extractor 2020-11-15 20:31:40 +01:00
selfisekai ede99f9f13 remove youtube:live, fix tests 2020-11-15 16:41:15 +01:00
selfisekai 92d1bd1b90 [youtube] brand new channel/playlist extractors 2020-11-15 15:44:07 +01:00
selfisekai 85d8ce7a3d Merge branch 'master' of ssh://git.sakamoto.pl:2137/laudom/haruhi-dl into youtube-sig-crypto-proposal 2020-11-12 11:04:15 +01:00
selfisekai 0259a32b73 [youtube] fix reworked sig decrypting 2020-11-12 11:02:07 +01:00
selfisekai 8c0ff392ea [youtube] dynamic sig improvements 2020-11-12 06:31:11 +01:00
Dominika 8c506dae76 version 2020.11.12 2020-11-12 02:53:28 +01:00
selfisekai e31c0d2576 [youtube] dynamic sig crypto fallback 2020-11-09 16:02:49 +01:00
selfisekai 835917442e [tiktok:user] only match user profiles 2020-11-08 21:52:02 +01:00
selfisekai 0958b54441 [wykop] add new extractor 2020-11-08 21:41:50 +01:00
selfisekai 5c6bcbf172 fixed tiktok video extractor 2020-11-08 05:37:01 +01:00
Dominika 7e9e7a3c91 version 2020.11.08 2020-11-08 00:40:17 +01:00
selfisekai ca3abff9ea aliexpress product video extractor 2020-11-07 04:13:19 +01:00
Dominika 84ea897ae8 version 2020.11.06 2020-11-06 23:30:05 +01:00
selfisekai 20d6ad3c33 [onnetwork] added extractor 2020-11-04 23:55:41 +01:00
Dominika 60ba8b14d6 I have no idea what I'm doing 2020-11-04 01:18:17 +01:00
selfisekai 6f623e3cd0 [cda] major fixes 2020-11-01 22:52:39 +01:00
Dominika 7fb9bc8b62 removed useless code, fixed URLs, fixed like/dislike count in youtube extractor 2020-11-01 22:24:53 +01:00
Dominika 5be3f4bd7f Merge branch 'master' of ssh://git.sakamoto.pl:2137/laudom/youtube-dl into master 2020-11-01 05:17:30 +01:00
Dominika 1c397d1dd3 version 2020.11.01 2020-11-01 05:17:15 +01:00
selfisekai a741425af8 [tvp] quick fixes 2020-11-01 05:03:15 +01:00
Dominika eacab4c6a3 bulk project name change, expect bugssssss 2020-11-01 04:39:17 +01:00