Repository navigation
get_clean_html: lxml error #108
Description
dufferzafar
Issue body actions
Reactions are currently unavailable
Activity
The
doc.summary()method works, but it doesn't seem to have all the data that we want.And I think I've found the error.
Line 9 of the file should actually be:
return clean_attributes(tounicode(self._html(True)))
So that it forces the
self.htmlattribute to be set.Gonna roll with this change for now.
get_clean_html is made for summary cleaning up, not for the full doc cleaning up.
You can clean the full doc yourself, you don't need this lib for that ;)I get the same error even if i used get_clean_html only on requests.get(url).text, but only on a spécifique url :https://start.lesechos.fr/travailler-a-letranger/actu-internationales/expatriation-les-pays-qui-chouchoutent-le-plus-les-talents-13988.php
page_response = requests.get(page_link) doc = Document(page_response.text) doc.get_clean_html()
--------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-36-f64b1934a8ea> in <module> 1 page_response = requests.get(page_link) 2 doc = Document(page_response.text) ----> 3 doc.get_clean_html() C:\ProgramData\Anaconda3\lib\site-packages\readability\readability.py in get_clean_html(self) 165 to disable or to improve DOM-to-text conversion in .summary() method 166 """ --> 167 return clean_attributes(tounicode(self.html)) 168 169 def summary(self, html_partial=False): src/lxml/etree.pyx in lxml.etree.tounicode() TypeError: Type '<class 'NoneType'>' cannot be serialized.
