Masked arrays add-ons.
A collection of utilities for numpy.ma.
| Author | |
| Pierre Gerard-Marchant | |
| Unknown Field: contact | |
| pierregm_at_uga_dot_edu | |
| Unknown Field: version | |
| $Id: extras.py 3473 2007-10-29 15:18:13Z jarrod.millman $ |
| Variable | atleast_1d |
Undocumented |
| Variable | atleast_2d |
Undocumented |
| Variable | atleast_3d |
Undocumented |
| Variable | column_stack |
Undocumented |
| Variable | diagflat |
Undocumented |
| Variable | dstack |
Undocumented |
| Variable | hsplit |
Undocumented |
| Variable | hstack |
Undocumented |
| Variable | mr_ |
Undocumented |
| Variable | row_stack |
Undocumented |
| Variable | stack |
Undocumented |
| Variable | vstack |
Undocumented |
| Class | _fromnxfunction |
Defines a wrapper to adapt NumPy functions to masked arrays. |
| Class | _fromnxfunction_allargs |
No summary |
| Class | _fromnxfunction_args |
No summary |
| Class | _fromnxfunction_seq |
A version of _fromnxfunction that is called with a single sequence of arrays followed by auxiliary args that are passed verbatim for both the data and mask calls. |
| Class | _fromnxfunction_single |
A version of _fromnxfunction that is called with a single array argument followed by auxiliary args that are passed verbatim for both the data and mask calls. |
| Class | MAxisConcatenator |
Translate slice objects to concatenation along an axis. |
| Class | mr_class |
Translate slice objects to concatenation along the first axis. |
| Function | _covhelper |
Private function for the computation of covariance and correlation coefficients. |
| Function | _ezclump |
Finds the clumps (groups of data with the same values) for a 1D bool array. |
| Function | _median |
Undocumented |
| Function | apply_along_axis |
(This docstring should be overwritten) |
| Function | apply_over_axes |
(This docstring will be overwritten) |
| Function | average |
Return the weighted average of array over the given axis. |
| Function | clump_masked |
Returns a list of slices corresponding to the masked clumps of a 1-D array. (A "clump" is defined as a contiguous region of the array). |
| Function | clump_unmasked |
Return list of slices corresponding to the unmasked clumps of a 1-D array. (A "clump" is defined as a contiguous region of the array). |
| Function | compress_cols |
Suppress whole columns of a 2-D array that contain masked values. |
| Function | compress_nd |
Suppress slices from multiple dimensions which contain masked values. |
| Function | compress_rowcols |
Suppress the rows and/or columns of a 2-D array that contain masked values. |
| Function | compress_rows |
Suppress whole rows of a 2-D array that contain masked values. |
| Function | corrcoef |
Return Pearson product-moment correlation coefficients. |
| Function | count_masked |
Count the number of masked elements along the given axis. |
| Function | cov |
Estimate the covariance matrix. |
| Function | dot |
Return the dot product of two arrays. |
| Function | ediff1d |
Compute the differences between consecutive elements of an array. |
| Function | flatnotmasked_contiguous |
Find contiguous unmasked data in a masked array along the given axis. |
| Function | flatnotmasked_edges |
Find the indices of the first and last unmasked values. |
| Function | flatten_inplace |
Flatten a sequence in place. |
| Function | in1d |
Test whether each element of an array is also present in a second array. |
| Function | intersect1d |
Returns the unique elements common to both arrays. |
| Function | isin |
Calculates element in test_elements, broadcasting over element only. |
| Function | issequence |
Is seq a sequence (ndarray, list or tuple)? |
| Function | mask_cols |
Mask columns of a 2D array that contain masked values. |
| Function | mask_rowcols |
Mask rows and/or columns of a 2D array that contain masked values. |
| Function | mask_rows |
Mask rows of a 2D array that contain masked values. |
| Function | masked_all |
Empty masked array with all elements masked. |
| Function | masked_all_like |
Empty masked array with the properties of an existing array. |
| Function | median |
Compute the median along the specified axis. |
| Function | notmasked_contiguous |
Find contiguous unmasked data in a masked array along the given axis. |
| Function | notmasked_edges |
Find the indices of the first and last unmasked values along an axis. |
| Function | polyfit |
Any masked values in x is propagated in y, and vice-versa. |
| Function | setdiff1d |
Set difference of 1D arrays with unique elements. |
| Function | setxor1d |
Set exclusive-or of 1-D arrays with unique elements. |
| Function | union1d |
Union of two arrays. |
| Function | unique |
Finds the unique elements of an array. |
| Function | vander |
Masked values in the input array result in rows of zeros. |
Finds the clumps (groups of data with the same values) for a 1D bool array.
Returns a series of slices.
Return the weighted average of array over the given axis.
a. If None, averaging is done over
the flattened array.The importance that each element has in the computation of the average.
The weights array can either be 1-D (in which case its length must be
the size of a along the given axis) or of the same shape as a.
If weights=None, then all data in a are assumed to have a
weight equal to one. The 1-D calculation is:
avg = sum(a * weights) / sum(weights)
The only constraint on weights is that sum(weights) must not be 0.
True,
return a tuple with the average as the first element and the sum
of the weights as the second element. The return type is np.float64
if a is of integer type and floats smaller than float64, or the
input data-type, otherwise. If returned, sum_of_weights is always
float64.>>> a = np.ma.array([1., 2., 3., 4.], mask=[False, False, True, True]) >>> np.ma.average(a, weights=[3, 1, 0, 0]) 1.25
>>> x = np.ma.arange(6.).reshape(3, 2) >>> x masked_array( data=[[0., 1.], [2., 3.], [4., 5.]], mask=False, fill_value=1e+20) >>> avg, sumweights = np.ma.average(x, axis=0, weights=[1, 2, 3], ... returned=True) >>> avg masked_array(data=[2.6666666666666665, 3.6666666666666665], mask=[False, False], fill_value=1e+20)
Returns a list of slices corresponding to the masked clumps of a 1-D array. (A "clump" is defined as a contiguous region of the array).
a.flatnotmasked_edges, flatnotmasked_contiguous, notmasked_edges notmasked_contiguous, clump_unmasked
>>> a = np.ma.masked_array(np.arange(10)) >>> a[[0, 1, 2, 6, 8, 9]] = np.ma.masked >>> np.ma.clump_masked(a) [slice(0, 3, None), slice(6, 7, None), slice(8, 10, None)]
Return list of slices corresponding to the unmasked clumps of a 1-D array. (A "clump" is defined as a contiguous region of the array).
a.flatnotmasked_edges, flatnotmasked_contiguous, notmasked_edges notmasked_contiguous, clump_masked
>>> a = np.ma.masked_array(np.arange(10)) >>> a[[0, 1, 2, 6, 8, 9]] = np.ma.masked >>> np.ma.clump_unmasked(a) [slice(3, 6, None), slice(7, 8, None)]
Suppress whole columns of a 2-D array that contain masked values.
This is equivalent to np.ma.compress_rowcols(a, 1), see
compress_rowcols for details.
compress_rowcols
Suppress slices from multiple dimensions which contain masked values.
x is interpreted as a MaskedArray with mask
set to nomask.Suppress the rows and/or columns of a 2-D array that contain masked values.
The suppression behavior is selected with the axis parameter.
x is interpreted as a MaskedArray with
mask set to nomask. Must be a 2D array.>>> x = np.ma.array(np.arange(9).reshape(3, 3), mask=[[1, 0, 0], ... [1, 0, 0], ... [0, 0, 0]]) >>> x masked_array( data=[[--, 1, 2], [--, 4, 5], [6, 7, 8]], mask=[[ True, False, False], [ True, False, False], [False, False, False]], fill_value=999999)
>>> np.ma.compress_rowcols(x) array([[7, 8]]) >>> np.ma.compress_rowcols(x, 0) array([[6, 7, 8]]) >>> np.ma.compress_rowcols(x, 1) array([[1, 2], [4, 5], [7, 8]])
Suppress whole rows of a 2-D array that contain masked values.
This is equivalent to np.ma.compress_rowcols(a, 0), see
compress_rowcols for details.
compress_rowcols
Return Pearson product-moment correlation coefficients.
Except for the handling of missing data this function does the same as
numpy.corrcoef. For more details and examples, see numpy.corrcoef.
x represents a variable, and each column a single
observation of all those variables. Also see rowvar below.y has the same
shape as x.rowvar is True (default), then each row represents a
variable, with observations in the columns. Otherwise, the relationship
is transposed: each column represents a variable, while the rows
contain observations.Has no effect, do not use.
x, the corresponding value is masked in y.
If False, raises an exception. Because bias is deprecated, this
argument needs to be treated as keyword only to avoid a warning.Has no effect, do not use.
numpy.corrcoef : Equivalent function in top-level NumPy module. cov : Estimate the covariance matrix.
This function accepts but discards arguments bias and ddof. This is
for backwards compatibility with previous versions of this function. These
arguments had no effect on the return values of the function and can be
safely ignored in this and previous versions of numpy.
Count the number of masked elements along the given axis.
MaskedArray.count : Count non-masked elements.
>>> import numpy.ma as ma >>> a = np.arange(9).reshape((3,3)) >>> a = ma.array(a) >>> a[1, 0] = ma.masked >>> a[1, 2] = ma.masked >>> a[2, 1] = ma.masked >>> a masked_array( data=[[0, 1, 2], [--, 4, --], [6, --, 8]], mask=[[False, False, False], [ True, False, True], [False, True, False]], fill_value=999999) >>> ma.count_masked(a) 3
When the axis keyword is used an array is returned.
>>> ma.count_masked(a, axis=0) array([1, 1, 1]) >>> ma.count_masked(a, axis=1) array([0, 2, 1])
Estimate the covariance matrix.
Except for the handling of missing data this function does the same as
numpy.cov. For more details and examples, see numpy.cov.
By default, masked values are recognized as such. If x and y have the
same shape, a common mask is allocated: if x[i,j] is masked, then
y[i,j] will also be masked.
Setting allow_masked to False will raise an exception if values are
missing in either of the input arrays.
x represents a variable, and each column a single
observation of all those variables. Also see rowvar below.y has the same
shape as x.rowvar is True (default), then each row represents a
variable, with observations in the columns. Otherwise, the relationship
is transposed: each column represents a variable, while the rows
contain observations.bias is True,
then normalization is by N. This keyword can be overridden by
the keyword ddof in numpy versions >= 1.5.x, the corresponding value is masked in y.
If False, raises a ValueError exception when some values are missing.If not None normalization is by (N - ddof), where N is the number of observations; this overrides the value implied by bias. The default value is None.
allow_masked is False.numpy.cov
Return the dot product of two arrays.
This function is the equivalent of numpy.dot that takes masked values
into account. Note that strict and out are in different position
than in the method version. In order to maintain compatibility with the
corresponding method, it is recommended that the optional arguments be
treated as keyword only. At some point that may be mandatory.
Note
Works only with 2-D arrays at the moment.
Output argument. This must have the exact kind that would be returned
if it was not used. In particular, it must have the right type, must be
C-contiguous, and its dtype must be the dtype that would be returned
for dot(a,b). This is a performance feature. Therefore, if these
conditions are not met, an exception is raised, instead of attempting
to be flexible.
numpy.dot : Equivalent function for ndarrays.
>>> a = np.ma.array([[1, 2, 3], [4, 5, 6]], mask=[[1, 0, 0], [0, 0, 0]]) >>> b = np.ma.array([[1, 2], [3, 4], [5, 6]], mask=[[1, 0], [0, 0], [0, 0]]) >>> np.ma.dot(a, b) masked_array( data=[[21, 26], [45, 64]], mask=[[False, False], [False, False]], fill_value=999999) >>> np.ma.dot(a, b, strict=True) masked_array( data=[[--, --], [--, 64]], mask=[[ True, True], [ True, False]], fill_value=999999)
Compute the differences between consecutive elements of an array.
This function is the equivalent of numpy.ediff1d that takes masked
values into account, see numpy.ediff1d for details.
numpy.ediff1d : Equivalent function for ndarrays.
Find contiguous unmasked data in a masked array along the given axis.
A sorted sequence of slice objects (start index, end index).
flatnotmasked_edges, notmasked_contiguous, notmasked_edges clump_masked, clump_unmasked
Only accepts 2-D arrays at most.
>>> a = np.ma.arange(10) >>> np.ma.flatnotmasked_contiguous(a) [slice(0, 10, None)]
>>> mask = (a < 3) | (a > 8) | (a == 5) >>> a[mask] = np.ma.masked >>> np.array(a[~a.mask]) array([3, 4, 6, 7, 8])
>>> np.ma.flatnotmasked_contiguous(a) [slice(3, 5, None), slice(6, 9, None)] >>> a[:] = np.ma.masked >>> np.ma.flatnotmasked_contiguous(a) []
Find the indices of the first and last unmasked values.
Expects a 1-D MaskedArray, returns None if all values are masked.
MaskedArrayflatnotmasked_contiguous, notmasked_contiguous, notmasked_edges clump_masked, clump_unmasked
Only accepts 1-D arrays.
>>> a = np.ma.arange(10) >>> np.ma.flatnotmasked_edges(a) array([0, 9])
>>> mask = (a < 3) | (a > 8) | (a == 5) >>> a[mask] = np.ma.masked >>> np.array(a[~a.mask]) array([3, 4, 6, 7, 8])
>>> np.ma.flatnotmasked_edges(a) array([3, 8])
>>> a[:] = np.ma.masked >>> print(np.ma.flatnotmasked_edges(a)) None
Test whether each element of an array is also present in a second array.
The output is always a masked array. See numpy.in1d for more details.
We recommend using isin instead of in1d for new code.
isin : Version of this function that preserves the shape of ar1. numpy.in1d : Equivalent function for ndarrays.
Returns the unique elements common to both arrays.
Masked values are considered equal one to the other. The output is always a masked array.
See numpy.intersect1d for more details.
numpy.intersect1d : Equivalent function for ndarrays.
>>> x = np.ma.array([1, 3, 3, 3], mask=[0, 0, 0, 1]) >>> y = np.ma.array([3, 1, 1, 1], mask=[0, 0, 0, 1]) >>> np.ma.intersect1d(x, y) masked_array(data=[1, 3, --], mask=[False, False, True], fill_value=999999)
Calculates element in test_elements, broadcasting over
element only.
The output is always a masked array of the same shape as element.
See numpy.isin for more details.
in1d : Flattened version of this function. numpy.isin : Equivalent function for ndarrays.
Mask columns of a 2D array that contain masked values.
This function is a shortcut to mask_rowcols with axis equal to 1.
mask_rowcols : Mask rows and/or columns of a 2D array. masked_where : Mask where a condition is met.
>>> import numpy.ma as ma >>> a = np.zeros((3, 3), dtype=int) >>> a[1, 1] = 1 >>> a array([[0, 0, 0], [0, 1, 0], [0, 0, 0]]) >>> a = ma.masked_equal(a, 1) >>> a masked_array( data=[[0, 0, 0], [0, --, 0], [0, 0, 0]], mask=[[False, False, False], [False, True, False], [False, False, False]], fill_value=1) >>> ma.mask_cols(a) masked_array( data=[[0, --, 0], [0, --, 0], [0, --, 0]], mask=[[False, True, False], [False, True, False], [False, True, False]], fill_value=1)
Mask rows and/or columns of a 2D array that contain masked values.
Mask whole rows and/or columns of a 2D array that contain
masked values. The masking behavior is selected using the
axis parameter.
- If
axisis None, rows and columns are masked.- If
axisis 0, only rows are masked.- If
axisis 1 or -1, only columns are masked.
mask set
to nomask (False). Must be a 2D array.axis parameter.a is not 2D.mask_rows : Mask rows of a 2D array that contain masked values. mask_cols : Mask cols of a 2D array that contain masked values. masked_where : Mask where a condition is met.
The input array's mask is modified by this function.
>>> import numpy.ma as ma >>> a = np.zeros((3, 3), dtype=int) >>> a[1, 1] = 1 >>> a array([[0, 0, 0], [0, 1, 0], [0, 0, 0]]) >>> a = ma.masked_equal(a, 1) >>> a masked_array( data=[[0, 0, 0], [0, --, 0], [0, 0, 0]], mask=[[False, False, False], [False, True, False], [False, False, False]], fill_value=1) >>> ma.mask_rowcols(a) masked_array( data=[[0, --, 0], [--, --, --], [0, --, 0]], mask=[[False, True, False], [ True, True, True], [False, True, False]], fill_value=1)
Mask rows of a 2D array that contain masked values.
This function is a shortcut to mask_rowcols with axis equal to 0.
mask_rowcols : Mask rows and/or columns of a 2D array. masked_where : Mask where a condition is met.
>>> import numpy.ma as ma >>> a = np.zeros((3, 3), dtype=int) >>> a[1, 1] = 1 >>> a array([[0, 0, 0], [0, 1, 0], [0, 0, 0]]) >>> a = ma.masked_equal(a, 1) >>> a masked_array( data=[[0, 0, 0], [0, --, 0], [0, 0, 0]], mask=[[False, False, False], [False, True, False], [False, False, False]], fill_value=1)
>>> ma.mask_rows(a) masked_array( data=[[0, 0, 0], [--, --, --], [0, 0, 0]], mask=[[False, False, False], [ True, True, True], [False, False, False]], fill_value=1)
Empty masked array with all elements masked.
Return an empty masked array of the given shape and dtype, where all the data are masked.
masked_all_like : Empty masked array modelled on an existing array.
>>> import numpy.ma as ma >>> ma.masked_all((3, 3)) masked_array( data=[[--, --, --], [--, --, --], [--, --, --]], mask=[[ True, True, True], [ True, True, True], [ True, True, True]], fill_value=1e+20, dtype=float64)
The dtype parameter defines the underlying data type.
>>> a = ma.masked_all((3, 3)) >>> a.dtype dtype('float64') >>> a = ma.masked_all((3, 3), dtype=np.int32) >>> a.dtype dtype('int32')
Empty masked array with the properties of an existing array.
Return an empty masked array of the same shape and dtype as
the array arr, where all the data are masked.
arr doesn't have a shape attribute (i.e. not an ndarray)masked_all : Empty masked array with all elements masked.
>>> import numpy.ma as ma >>> arr = np.zeros((2, 3), dtype=np.float32) >>> arr array([[0., 0., 0.], [0., 0., 0.]], dtype=float32) >>> ma.masked_all_like(arr) masked_array( data=[[--, --, --], [--, --, --]], mask=[[ True, True, True], [ True, True, True]], fill_value=1e+20, dtype=float32)
The dtype of the masked array matches the dtype of arr.
>>> arr.dtype dtype('float32') >>> ma.masked_all_like(arr).dtype dtype('float32')
Compute the median along the specified axis.
Returns the median of the array elements.
overwrite_input is True, and the input
is not already an ndarray, an error will be raised.If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input array.
mean
Given a vector V with N non masked values, the median of V is the middle value of a sorted copy of V (Vs) - i.e. Vs[(N-1)/2], when N is odd, or {Vs[N/2 - 1] + Vs[N/2]}/2 when N is even.
>>> x = np.ma.array(np.arange(8), mask=[0]*4 + [1]*4) >>> np.ma.median(x) 1.5
>>> x = np.ma.array(np.arange(10).reshape(2, 5), mask=[0]*6 + [1]*4) >>> np.ma.median(x) 2.5 >>> np.ma.median(x, axis=-1, overwrite_input=True) masked_array(data=[2.0, 5.0], mask=[False, False], fill_value=1e+20)
Find contiguous unmasked data in a masked array along the given axis.
flatnotmasked_contiguous.A list of slices (start and end indexes) of unmasked indexes in the array.
If the input is 2d and axis is specified, the result is a list of lists.
flatnotmasked_edges, flatnotmasked_contiguous, notmasked_edges clump_masked, clump_unmasked
Only accepts 2-D arrays at most.
>>> a = np.arange(12).reshape((3, 4)) >>> mask = np.zeros_like(a) >>> mask[1:, :-1] = 1; mask[0, 1] = 1; mask[-1, 0] = 0 >>> ma = np.ma.array(a, mask=mask) >>> ma masked_array( data=[[0, --, 2, 3], [--, --, --, 7], [8, --, --, 11]], mask=[[False, True, False, False], [ True, True, True, False], [False, True, True, False]], fill_value=999999) >>> np.array(ma[~ma.mask]) array([ 0, 2, 3, 7, 8, 11])
>>> np.ma.notmasked_contiguous(ma) [slice(0, 1, None), slice(2, 4, None), slice(7, 9, None), slice(11, 12, None)]
>>> np.ma.notmasked_contiguous(ma, axis=0) [[slice(0, 1, None), slice(2, 3, None)], [], [slice(0, 1, None)], [slice(0, 3, None)]]
>>> np.ma.notmasked_contiguous(ma, axis=1) [[slice(0, 1, None), slice(2, 4, None)], [slice(3, 4, None)], [slice(0, 1, None), slice(3, 4, None)]]
Find the indices of the first and last unmasked values along an axis.
If all values are masked, return None. Otherwise, return a list of two tuples, corresponding to the indices of the first and last unmasked values respectively.
edges is a
list of the first and last index.flatnotmasked_contiguous, flatnotmasked_edges, notmasked_contiguous clump_masked, clump_unmasked
>>> a = np.arange(9).reshape((3, 3)) >>> m = np.zeros_like(a) >>> m[1:, 1:] = 1
>>> am = np.ma.array(a, mask=m) >>> np.array(am[~am.mask]) array([0, 1, 2, 3, 6])
>>> np.ma.notmasked_edges(am) array([0, 6])
Set difference of 1D arrays with unique elements.
The output is always a masked array. See numpy.setdiff1d for more
details.
numpy.setdiff1d : Equivalent function for ndarrays.
>>> x = np.ma.array([1, 2, 3, 4], mask=[0, 1, 0, 1]) >>> np.ma.setdiff1d(x, [1, 2]) masked_array(data=[3, --], mask=[False, True], fill_value=999999)
Set exclusive-or of 1-D arrays with unique elements.
The output is always a masked array. See numpy.setxor1d for more details.
numpy.setxor1d : Equivalent function for ndarrays.
Union of two arrays.
The output is always a masked array. See numpy.union1d for more details.
numpy.union1d : Equivalent function for ndarrays.