Théophile Bastian
c3bcdea1eb
Add tentative export to RDF
2018-02-25 14:37:30 +01:00
Théophile Bastian
2732e4115f
Add RDF models export classes — untested
...
Also add a dependency to https://github.com/tobast/RDFSerializer/
2018-02-23 13:32:32 +01:00
Rémi Oudin
0e02f22d08
Exception handling
...
Big problem with the url https:/plus.google.com/+Python concerning
robots parsing.
Didn't find the bug. @tobast , if you have some time to look at it :)
2018-02-23 00:37:36 +01:00
Rémi Oudin
77ca7ebcb9
Silly me.
2018-02-22 15:35:46 +01:00
Rémi Oudin
9b78e268c9
Nearly working crawler
2018-02-22 14:33:07 +01:00
Rémi Oudin
e19e623df1
Multiple bug fixes. TODO : remove <div id=footer>-like patterns
2018-02-22 14:07:53 +01:00
Rémi Oudin
236e15296c
It can be useful to return the links list
2018-02-21 23:11:57 +01:00
Rémi Oudin
4e6ac5ac7b
Url getter function : retrieves the list of so-called relevant links
2018-02-21 22:51:05 +01:00
Rémi Oudin
a907cad33d
Start of url getter function
2018-02-21 19:06:46 +01:00
Théophile Bastian
b05e642c79
Make the code somewhat readable
2018-02-21 11:54:41 +01:00
Théophile Bastian
c97acb22b5
Add tentative crawl file
...
Nothing functional, just tests
2018-02-20 12:48:53 +01:00
Théophile Bastian
c05c2561d2
Add crawler settings and requirements
2018-02-20 12:48:16 +01:00
Théophile Bastian
bef1fca5b9
Init app 'crawl'
2018-02-20 08:51:16 +01:00
Théophile Bastian
924657abdb
Generate profiles' migration
2018-01-24 22:49:34 +01:00
Théophile Bastian
e9b3127226
Use `profiles` as an installed application in pinocchio
2018-01-24 22:49:08 +01:00
Théophile Bastian
cbf1911fe7
Add models for Interest and Profile
2018-01-24 22:48:53 +01:00
Théophile Bastian
37581fb96a
Add models for Place and Event
2018-01-24 22:39:20 +01:00
Théophile Bastian
6531415d63
Add model for a webpage and website
2018-01-24 14:09:33 +01:00
Théophile Bastian
114c8a3d3e
Add model for search engines
2018-01-24 13:52:43 +01:00
Théophile Bastian
225742798b
Add BrowserFingerprint model
2018-01-24 13:36:55 +01:00
Théophile Bastian
a3e6308837
Init apps `histories` and `profiles`
2018-01-23 18:12:47 +01:00
Théophile Bastian
397784a673
Add first version of requirements.txt
...
Mainly Django, by now
2018-01-23 18:11:07 +01:00
Théophile Bastian
132b7250c8
Initialize Django
2018-01-23 18:11:00 +01:00
Théophile Bastian
c1e3be346f
Initial commit
2018-01-23 17:53:08 +01:00
Théophile Bastian
Rémi Oudin