Commit graph

23 commits

Author SHA1 Message Date
98fe69ba62 Real async crawling 2018-02-26 15:30:38 +01:00
968ff6d24c More robust crawling 2018-02-26 15:29:36 +01:00
5d4bd30e20 Exception handling 2018-02-26 15:15:03 +01:00
236e40d359 Sanity check 2018-02-26 14:57:46 +01:00
549c861908 Bug fixé 2018-02-26 14:38:26 +01:00
c4f63a92b2 Error in the merge, mea culpa 2018-02-26 14:01:29 +01:00
45ddbff91a Crawling and histories: fix a lot of stuff 2018-02-26 11:49:24 +01:00
e6d587bffd Actually save to DB a created history 2018-02-26 11:49:24 +01:00
22064ebee3 Histories: xml import/export — untested
To be tested when history generation is available
2018-02-26 11:48:51 +01:00
a4de51b84a Crawl: do not use global SEARCH_ENGINES 2018-02-26 11:48:51 +01:00
4a8bd32516 Fix tor_runner import 2018-02-26 11:48:51 +01:00
adb892ab7d Check if crawling a search engine 2018-02-26 11:12:36 +01:00
d6b26c0a46 Better use of history 2018-02-26 10:05:33 +01:00
7c0fb7dda1 Better naming 2018-02-25 11:49:44 +01:00
bc7348f677 Integration of crawl module in histories 2018-02-24 23:17:24 +01:00
854481dbd3 Import utilities 2018-02-24 17:21:41 +01:00
5decd205fb Typos + improvements 2018-02-22 11:06:45 +01:00
cd4d8a4c3f More generic code using @8f4458b 2018-02-21 11:50:28 +01:00
4920de5838 Going on in the generation of history 2018-02-20 23:42:21 +01:00
7c13ee17d4 Skeleton of history generation 2018-02-19 22:56:16 +01:00
7f343d8ad8 Better formatting 2018-02-19 13:59:29 +01:00
60f09bd4d3 Add basic models for histories 2018-02-19 13:58:55 +01:00
a3e6308837 Init apps histories and profiles 2018-01-23 18:12:47 +01:00