80 commits

Author	SHA1	Message	Date
Jan Lukas Gernert	2460745547	cleanup	2023-02-25 00:44:18 +01:00
Jan Lukas Gernert	63035ca028	fmt	2023-02-25 00:43:42 +01:00
Jan Lukas Gernert	e3246af28b	refactor & more testing	2023-02-25 00:42:26 +01:00
Jan Lukas Gernert	7ae98904d4	unwrap noscript images	2023-02-23 01:53:42 +01:00
Jan Lukas Gernert	98c06e11f4	improve title extraction	2023-02-20 02:32:58 +01:00
Jan Lukas Gernert	cce912c354	first content extraction kinda working	2023-02-20 00:29:44 +01:00
Jan Lukas Gernert	2c76a869e7	fmt	2023-02-17 14:35:35 +01:00
Jan Lukas Gernert	71a8816747	somewhat complete readability algorithm	2023-02-17 14:16:01 +01:00
Jan Lukas Gernert	979358fd35	more	2023-01-01 21:35:46 +01:00
Jan Lukas Gernert	2750ad648d	start implementing readability	2023-01-01 14:51:34 +01:00
Jan Lukas Gernert	c08f5afa5d	move stuff around	2022-12-13 08:54:57 +01:00
Jan Lukas Gernert	90383545e0	extract & parse charsets other than utf8	2022-12-11 17:38:42 +01:00
Jan Lukas Gernert	97b194c9e8	clippy regex escape	2022-12-11 16:31:01 +01:00
Jan Lukas Gernert	88bb88a38f	clippy	2022-12-11 16:23:02 +01:00
Jan Lukas Gernert	dc1bf2ef0c	fmt	2022-12-11 16:19:49 +01:00
Jan Lukas Gernert	22e98fdab7	extract thumbnail url	2022-12-11 16:18:03 +01:00
Jan Lukas Gernert	0c8aba4f4a	refactor: a bit less nested code	2022-12-01 10:14:47 +01:00
Jan Lukas Gernert	27be5a3204	port failure -> thiserror	2022-12-01 09:22:08 +01:00
Jan Lukas Gernert	d906f6b7fe	readability stub	2022-10-08 23:10:26 +02:00
Jan Lukas Gernert	273ddd832c	start refactor & fingerprints	2022-10-08 23:09:00 +02:00
Jan Lukas Gernert	7b205e8e27	fmt	2022-10-07 09:32:39 +02:00
Jan Lukas Gernert	69659da983	clippy fixes	2022-10-07 09:20:10 +02:00
Jan Lukas Gernert	8c2af14871	special handling trying to find single page links: fixes youtube	2022-10-07 08:48:09 +02:00
Jan Lukas Gernert	7b1b027c6d	add support for header values: fixes golem test	2022-10-07 07:17:33 +02:00
Jan Lukas Gernert	0e3553b647	remove dbg code	2022-10-07 07:17:33 +02:00
Jan Lukas Gernert	c1ae011fcd	use global rules	2022-10-07 07:17:31 +02:00
Jan Lukas Gernert	3a6a70ee64	embedded config files	2022-10-07 07:16:54 +02:00
Jan Lukas Gernert	aa09666f4c	async config loading	2022-10-07 07:16:06 +02:00
Jan Lukas Gernert	9fb772bfa8	update deps	2022-10-07 07:16:06 +02:00
Volker Weißmann	593901c849	Fixed spelling	2022-06-15 19:15:51 +02:00
Jan Lukas Gernert	76940232a5	take url reference	2021-01-21 08:53:51 +01:00
Jan Lukas Gernert	b73448b189	fix clippy lints	2021-01-06 10:32:43 +01:00
Jan Lukas Gernert	7e05a98f30	update to tokio 1.0	2021-01-06 09:53:47 +01:00
Jan Lukas Gernert	196a106e7a	shut up clippy	2020-06-07 13:40:08 +02:00
Jan Lukas Gernert	6b6c52f315	only use builtin youtube parsing if no config is provided	2020-06-07 13:21:53 +02:00
Jan Lukas Gernert	34eaf1eeb1	fmt	2020-06-07 12:53:33 +02:00
Jan Lukas Gernert	82a0a46323	special handling for youtube videos	2020-06-07 12:39:44 +02:00
Jan Lukas Gernert	a42ececb2a	check if final url differs from original even without redirect status	2020-06-06 05:18:25 +02:00
Felix Buehler	fa54b82e52	[ci] add fmt + lint checking	2020-05-30 13:07:10 +02:00
Felix Buehler	0c3946dd5b	fix fmt+lint	2020-05-29 18:55:00 +02:00
Jan Lukas Gernert	f51605a92c	naivedatetime -> datetime utc	2020-05-20 16:33:40 +02:00
Jan Lukas Gernert	8f48b69161	remove unneeded files	2020-04-28 03:07:21 +02:00
Jan Lukas Gernert	1fd7173eac	update for newer deps	2020-04-28 02:51:30 +02:00
Jan Lukas Gernert	d2960d8539	require client for parsing	2020-02-10 18:01:35 +01:00
Jan Lukas Gernert	1ecc0fc4b4	option to set custom reqwest client	2020-02-03 17:46:54 +01:00
Jan Lukas Gernert	71055eed1c	fix corrupt filename	2020-01-27 17:32:17 +01:00
Jan Lukas Gernert	98348b7e59	tmp: dont strip scripts	2020-01-27 16:47:13 +01:00
Jan Lukas Gernert	23514aff9e	less dramatic logging	2020-01-27 02:03:06 +01:00
Jan Lukas Gernert	afe661fe6c	only go for single page link if xpath res isn't empty	2020-01-27 01:54:37 +01:00
Jan Lukas Gernert	e58acf828c	improve logging clearity	2020-01-27 01:48:54 +01:00