ChirpText is a collection of text processing tools for Python 3.
It is not meant to be a powerful tank like the popular NTLK but a small package which you can pip-install anywhere and write a few lines of code to process textual data.
open()
function (txt, gz, binary, etc.)mecab-python3
package even on Windows, only a binary release (i.e. mecab.exe
) is required)texttaglib
TTL/CSV and TTL/JSON)chirptext
is available on PyPI and can be installed using pip
pip install chirptext
Note: chirptext library does not support Python 2 anymore. Please update to Python 3 to use this package.
You can download mecab binary package from http://taku910.github.io/mecab/#download and install it. After installed you can try:
>>> from chirptext import deko
>>> sent = deko.parse('猫が好きです。')
>>> sent.tokens
[[猫(名詞-一般/*/*|猫|ネコ|ネコ)], [が(助詞-格助詞/一般/*|が|ガ|ガ)], [好き(名詞-形容動詞語幹/*/*|好き|スキ|スキ)], [です(助動詞-*/*/*|です|デス|デス)], [。(記号-句点/*/*|。|。|。)], [EOS(-//|||)]]
>>> sent.words
['猫', 'が', '好き', 'です', '。']
>>> sent[0].pos
'名詞'
>>> sent[0].root
'猫'
>>> sent[0].reading
'ネコ'
If you installed MeCab to a custom location, for example C:\mecab\bin\mecab.exe
, try
>>> deko.set_mecab_bin("C:\\mecab\\bin\\mecab.exe")
>>> deko.get_mecab_bin()
'C:\\mecab\\bin\\mecab.exe'
# Just that & now you can use mecab
>>> deko.parse('雨が降る。').words
['雨', 'が', '降る', '。']
>>> from chirptext import chio
>>> chio.write_tsv('data/test.tsv', [['a', 'b'], ['c', 'd']])
>>> chio.read_tsv('data/tes.tsv')
[['a', 'b'], ['c', 'd']]
>>> chio.write_file('data/content.tar.gz', 'Support writing to .tar.gz file')
>>> chio.read_file('data/content.tar.gz')
'Support writing to .tar.gz file'
>>> for row in chio.read_tsv_iter('data/test.tsv'):
... print(row)
...
['a', 'b']
['c', 'd']
# a string report
rp = TextReport() # by default, TextReport will write to standard output, i.e. terminal
rp = TextReport(TextReport.STDOUT) # same as above
rp = TextReport('~/tmp/my-report.txt') # output to a file
rp = TextReport.null() # ouptut to /dev/null, i.e. nowhere
rp = TextReport.string() # output to a string. Call rp.content() to get the string
rp = TextReport(TextReport.STRINGIO) # same as above
# TextReport will close the output stream automatically by using the with statement
with TextReport.string() as rp:
rp.header("Lorem Ipsum Analysis", level="h0")
rp.header("Raw", level="h1")
rp.print(LOREM_IPSUM)
rp.header("Top 5 most common letters")
ct.summarise(report=rp, limit=5)
print(rp.content())
+----------------------------------------------------------------------------------
| Lorem Ipsum Analysis
+----------------------------------------------------------------------------------
Raw
------------------------------------------------------------
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
Top 5 most common letters
------------------------------------------------------------
i: 42
e: 37
t: 32
o: 29
a: 29
Author: letuananh
Live Demo: View The Demo
Download Link: Download The Source Code
Official Website: https://github.com/letuananh/chirptext
License: MIT
#python