xphyle is a small python (3.3+) library that makes it easy to open compressed files. Most importantly, xphyle will use the appropriate program (e.g. ‘gzip’) to compress/decompress a file if it is available on your system; this is almost always faster than using the corresponding python library. xphyle also provides methods that simplify common file I/O operations.
pip install xphyle
If you are using python 3.3 or 3.4, this will install one dependency: backports.typing.
Clone this repository and run
from xphyle import * from xphyle.paths import STDIN, STDOUT # Open a compressed file... myfile = xopen('infile.gz') # ...or a compressed stream # e.g. gzip -c afile | python my_program.py stdin = xopen(STDIN) # Easily write to the stdin of a subprocess with open_('|cat', 'wt') as process: process.write('foo') # We have to tell xopen what kind of compression # to use when writing to stdout stdout = xopen(STDOUT, compression='gz') # Print all lines in a compressed file... with open_('infile.gz') as myfile: for line in myfile: print(line) # ... or a compressed URL with open_('http://foo.com/myfile.gz') as myfile: for line in myfile: print(line) # Transparently handle paths and file objects def dostuff(path_or_file): with open_(path_or_file) as myfile: for line in myfile: print(line) # Read all lines in a compressed file into a list from xphyle.utils import read_lines lines = list(read_lines('infile.gz')) # Sum the rows in a compressed file where each line is an integer value total = sum(read_lines('infile.gz', convert=int))
See the Documentation for full usage information.
Future releases are mapped out using GitHub Projects.
We welcome any contributions via pull requests. Style-wise, we try to adhere to the Google python style guidelines. We use Google-style docstrings, which are formatted by the Napoleon Sphinx Plugin. We run pylint as part of each build and strive to maintain a 10/10 score. However, we disable some pylint checks: