README.md 3.72 KB
Newer Older
Harold Carrel Billiard's avatar
Harold Carrel Billiard committed
1 2 3
Converter : Wikipedia_to_Hdoc
---------------

Harold Carrel Billiard's avatar
Harold Carrel Billiard committed
4
Licence : 
Harold Carrel Billiard's avatar
Harold Carrel Billiard committed
5
---------------
Harold Carrel Billiard's avatar
Harold Carrel Billiard committed
6 7 8
GPL 3.0
http://www.gnu.org/licenses/gpl-3.0.txt

Harold Carrel Billiard's avatar
Harold Carrel Billiard committed
9 10 11 12 13 14 15 16 17 18 19

Credits :
---------------
Carrel Billiard Harold

Harriga Merouane

Lhomme Nicolas



20 21 22 23 24 25 26 27 28
Getting started 
---------------

Use a terminal and go to the root of the folder (Wikipedia_to_hdoc).

Generating .hdoc of a Wikipedia article with an URL
---------------------------------------------------

1 - Run the comand corresponding to your OS
Nicolas Lhomme's avatar
Nicolas Lhomme committed
29
        
30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
        On windows : 
            runURL.bat yourWikipediaUrl yourFilename
                yourWikipediaUrl is the Wikipedia URL
                yourFilename is the name of the directory in which output files will be placed
                
            For instance : runURL.bat https://fr.wikipedia.org/wiki/Constructeur_(programmation) constructeur
        
        On Linux : 
            sh runURL.sh yourWikipediaUrl yourFilename
                yourWikipediaUrl is the Wikipedia URL
                yourFilename is the name of the directory in which output files will be placed
            
            For instance : sh runURL.sh https://fr.wikipedia.org/wiki/Constructeur_(programmation) constructeur
            
2 - Get the .hdoc in the output/yourFilename folder

Generating .hdoc of a Wikipedia article with a local file
---------------------------------------------------------

lhommeni's avatar
lhommeni committed
49
1 - Copy the content of the Wikipedia article you want to convert in the directory named "input" and in a file called "source.xml".
Nicolas Lhomme's avatar
Nicolas Lhomme committed
50

51
2 - Run the comand corresponding to your OS
Nicolas Lhomme's avatar
Nicolas Lhomme committed
52 53 54
        
        
         windows : 
55 56 57 58 59
            runFile.bat
        
        On Linux : 
            sh runFile.sh
                       
60 61 62 63
3 - Get the .hdoc in the output/source folder

To do
---------------------------------------------------------
lhommeni's avatar
lhommeni committed
64
Concerning images :
65

lhommeni's avatar
lhommeni committed
66
1 - Extract the metadata information from the meta.xml file for each image. You can do that by creating an XSL file that will be called from the ant task generated by xslt/get_ressources_urls.xsl. In that file you have the hand on each meta.xml File.
Harold Carrel Billiard's avatar
Harold Carrel Billiard committed
67

68
2 - Verify that images are well zipped to avoid any problem while testing in Opale
Harold Carrel Billiard's avatar
Harold Carrel Billiard committed
69

70 71
3 - Images inside paragraphs break the validation of the hdoc schema, do a preposition to change the schema and handle that.

Harold Carrel Billiard's avatar
Harold Carrel Billiard committed
72

lhommeni's avatar
lhommeni committed
73 74 75 76
Concerning listings :

1 - Succeed in finding the language of the part of code of the wikipedia article

haroldcb's avatar
haroldcb committed
77 78 79 80

Concerning tables : 

1 - Solve the encoding problem,
Harold Carrel Billiard's avatar
Harold Carrel Billiard committed
81

haroldcb's avatar
haroldcb committed
82
2 - Change Hdoc Scheme in order to accept images in tables?
Harold Carrel Billiard's avatar
Harold Carrel Billiard committed
83 84 85

3 - Display complex tables as tables in Opale (not as extern files)

haroldcb's avatar
haroldcb committed
86
 
87 88
Be aware of the following things
---------------------------------------------------------
harriga_merouane@hotmail.fr's avatar
Merge  
harriga_merouane@hotmail.fr committed
89
1 - Not all images have a metadata information (only the ones who )
Nicolas Lhomme's avatar
Nicolas Lhomme committed
90
2 - The title of the images have a metadata information (only the ones who are not included in the text)
91 92 93

BUG
---
Harold Carrel Billiard's avatar
Harold Carrel Billiard committed
94 95 96 97 98 99 100 101
1 - Linux sh files doesn't work with UTC proxy but works outside of UTC.

2 - Random errors might occur
Wikipedia is a great tool : everyone can participe. However, it does not provide contributors with best practices that everyone follows. The result is a lot of different ways to write articles. This is why this converter might not handle some situations (even if all files I have tried worked), and it might not be able to output some Wikipedia articles at its current state.

3 - Small issues with Opale
Links can be invisible if you use an old version of Opale. This is not a problem coming from the Wikipedia to Hdoc converter. Make sure you use an updated version of Opale to test your scar archives.
Another thing is that Opale might indicate that the scar file contains errors once imported. Actually, these "errors" are warnings. The archives work, as they were validated when making the scar file. These warnings come from Opale, but you can ignore them.