numpy.lib._datasource

module documentation

A file interface for handling local and remote data files.

The goal of datasource is to abstract some of the file system operations when dealing with data files so the researcher doesn't have to know all the low-level details. Through datasource, a researcher can obtain and use a file with one function call, regardless of location of the file.

DataSource is meant to augment standard python libraries, not replace them. It should work seamlessly with standard file IO operations and the os module.

DataSource files can originate locally or remotely:

local files : '/home/guido/src/local/data.txt'
URLs (http, ftp, ...) : 'http://www.scipy.org/not/real/data.txt'

DataSource files can also be compressed or uncompressed. Currently only gzip, bz2 and xz are supported.

Example:

>>> # Create a DataSource, use os.curdir (default) for local storage.
>>> from numpy import DataSource
>>> ds = DataSource()
>>>
>>> # Open a remote file.
>>> # DataSource downloads the file, stores it locally in:
>>> #     './www.google.com/index.html'
>>> # opens the file and returns a file object.
>>> fp = ds.open('http://www.google.com/') # doctest: +SKIP
>>>
>>> # Use the file as you normally would
>>> fp.read() # doctest: +SKIP
>>> fp.close() # doctest: +SKIP

Class	`_FileOpeners`	Container for different methods to open (un-)compressed files.
Class	`Repository`	Repository(baseurl, destpath='.')
Function	`_check_mode`	Check mode and that encoding and newline are compatible.
Function	`open`	Open `path` with `mode` and return the file object.
Variable	`_file_openers`	Undocumented

def _check_mode(mode, encoding, newline):

Check mode and that encoding and newline are compatible.

Parameters

mode : str: File open mode.
encoding : str: File encoding.
newline : str: Newline for text files.

def open(path, mode='r', destpath=os.curdir, encoding=None, newline=None):

Open path with mode and return the file object.

If path is an URL, it will be downloaded, stored in the DataSource destpath directory and opened from there.

Parameters

path : str: Local file path or URL to open.
mode : str, optional: Mode to open path. Mode 'r' for reading, 'w' for writing, 'a' to append. Available modes depend on the type of object specified by path. Default is 'r'.
destpath : str, optional: Path to the directory where the source file gets downloaded to for use. If destpath is None, a temporary directory will be created. The default path is the current directory.
encoding : {None, str}, optional: Open text file with given encoding. The default encoding will be what io.open uses.
newline : {None, str}, optional: Newline to use when reading text file.

Returns

out : file object: The opened file.

Notes

This is a convenience function that instantiates a DataSource and returns the file object from DataSource.open(path).

_file_openers =

Undocumented