AUTOMATIC MAIN TEXT EXTRACTION FROM WEB PAGES
Keywords:HTML, BTE, body text etraction, main content text
AbstractThis paper presents a novel method for extracting body text from web pages used for building text corpus. The algorithm for extracting body text proposed by Aidan Finn  is extended with some enhancements in this research. The experimental results on a set of websites show that the proposed method significantly improves the performance of body text extraction without decrease in accuracy compared to the original algorithm.
Authors who publish with Vietnam Journal of Science and Technology agree with the following terms:
- The manuscript is not under consideration for publication elsewhere. When a manuscript is accepted for publication, the author agrees to automatic transfer of the copyright to the editorial office.
- The manuscript should not be published elsewhere in any language without the consent of the copyright holders. Authors have the right to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal’s published version of their work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are encouraged to post their work online (e.g., in institutional repositories or on their websites) prior to or during the submission process, as it can lead to productive exchanges or/and greater number of citation to the to-be-published work (See The Effect of Open Access).