spacepy.pycdf.CDF¶
-
class
spacepy.pycdf.
CDF
(pathname, masterpath=None, create=None, readonly=None)[source]¶ Python object representing a CDF file.
Open or create a CDF file by creating an object of this class.
Parameters: pathname : string
name of the file to open or create
masterpath : string
name of the master CDF file to use in creating a new file. If not provided, an existing file is opened; if provided but evaluates to
False
(e.g.,''
), an empty new CDF is created.create : bool
Create a new CDF even if masterpath isn’t provided
readonly : bool
Open the CDF read-only. Default True if opening an existing CDF; False if creating a new one. A readonly CDF with many variables may be slow to close. See
readonly()
.Raises: CDFError
if CDF library reports an error
Warns: CDFWarning
if CDF library reports a warning and interpreter is set to error on warnings.
Examples
Open a CDF by creating a CDF object, e.g.:
>>> cdffile = pycdf.CDF('cdf_filename.cdf')
Be sure to
close()
orsave()
when done.Note
Existing CDF files are opened read-only by default, see
readonly()
to change.CDF supports the with keyword, like other file objects, so:
>>> with pycdf.CDF('cdf_filename.cdf') as cdffile: ... #do brilliant things with the CDF
will open the CDF, execute the indented statements, and close the CDF when finished or when an error occurs. The python docs include more detail on this ‘context manager’ ability.
CDF objects behave like a python dictionary, where the keys are names of variables in the CDF, and the values,
Var
objects. As a dictionary, they are also iterable and it is easy to loop over all of the variables in a file. Some examples:List the names of all variables in the open CDF
cdffile
:>>> cdffile.keys() >>> for k in cdffile: #Alternate ... print(k)
Get a
Var
object for the variable namedEpoch
:>>> epoch = cdffile['Epoch']
Determine if a CDF contains a variable named
B_GSE
:>>> if 'B_GSE' in cdffile: ... print('B_GSE is in the file') ... else: ... print('B_GSE is not in the file')
Find how many variables are in the file:
>>> print(len(cdffile))
Delete the variable
Epoch
from the open CDF filecdffile
:>>> del cdffile['Epoch']
Display a summary of variables and types in open CDF file
cdffile
:>>> print(cdffile)
Open the CDF named
cdf_filename.cdf
, read all the data from all variables into dictionarydata
, and close it when done or if an error occurs:>>> with pycdf.CDF('cdf_filename.cdf') as cdffile: ... data = cdffile.copy()
This last example can be very inefficient as it reads the entire CDF. Normally it’s better to treat the CDF as a dictionary and access only the data needed, which will be pulled transparently from disc. See
Var
for more subtle examples.Potentially useful dictionary methods and related functions:
The CDF user’s guide section 2.2 has more background information on CDF files.
The
attrs
Python attribute acts as a dictionary referencing CDF attributes (do not confuse the two); all the dictionary methods above also work on the attribute dictionary. SeegAttrList
for more on the dictionary of global attributes.Creating a new CDF from a master (skeleton) CDF has similar syntax to opening one:
>>> cdffile = pycdf.CDF('cdf_filename.cdf', 'master_cdf_filename.cdf')
This creates and opens
cdf_filename.cdf
as a copy ofmaster_cdf_filename.cdf
.Using a skeleton CDF is recommended over making a CDF entirely from scratch, but this is possible by specifying a blank master:
>>> cdffile = pycdf.CDF('cdf_filename.cdf', '')
When CDFs are created in this way, they are opened read-write, see
readonly()
to change.By default, new CDFs (without a master) are created in version 2 (backward-compatible) format. To create a version 3 CDF, use
Library.set_backward()
:>>> pycdf.lib.set_backward(False) >>> cdffile = pycdf.CDF('cdf_filename.cdf', '')
Add variables by direct assignment, which will automatically set type and dimension based on the data provided:
>>> cdffile['new_variable_name'] = [1, 2, 3, 4]
or, if more control is needed over the type and dimensions, use
new()
.Although it is supported to assign Var objects to Python variables for convenience, there are some minor pitfalls that can arise when changing a CDF that will not affect most users. This is only a concern when assigning a zVar object to a Python variable, changing the CDF through some other variable, and then trying to use the zVar object via the originally assigned variable.
Deleting a variable:
>>> var = cdffile['Var1'] >>> del cdffile['Var1'] >>> var[0] #fail, no such variable
Renaming a variable:
>>> var = cdffile['Var1'] >>> cdffile['Var1'].rename('Var2') >>> var[0] #fail, no such variable
Renaming via the same variable works:
>>> var = cdffile['Var1'] >>> var.rename('Var2') >>> var[0] #succeeds, aware of new name
Deleting a variable and then creating another variable with the same name may lead to some surprises:
>>> var = cdffile['Var1'] >>> var[...] = [1, 2, 3, 4] >>> del cdffile['Var1'] >>> cdffile.new('Var1', data=[5, 6, 7, 8] >>> var[...] [5, 6, 7, 8]
attr_num
(attrname)Get the attribute number and scope by attribute name attrs
Global attributes for this CDF in a dict-like format. add_attr_to_cache
(attrname, num, scope)Add an attribute to the name-to-number cache add_to_cache
(varname, num)Add a variable to the name-to-number cache CDF.backward
checksum
([new_val])Set or check the checksum status of this CDF. clear_attr_from_cache
(attrname)Mark an attribute deleted in the name-to-number cache clear_from_cache
(varname)Mark a variable deleted in the name-to-number cache clone
(zVar[, name, data])Clone a zVariable (from another CDF or this) into this CDF close
()Closes the CDF file col_major
([new_col])Finds the majority of this CDF file compress
([comptype, param])Set or check the compression of this CDF copy
()Make a copy of all data and attributes in this CDF from_data
(filename, sd)Create a new CDF file from a SpaceData object or similar new
(name[, data, type, recVary, dimVarys, …])Create a new zVariable in this CDF raw_var
(name)Get a “raw” Var
object.readonly
([ro])Sets or check the readonly status of this CDF save
()Saves the CDF file but leaves it open. var_num
(varname)Get the variable number of a particular variable name version
()Get version of library that created this CDF -
backward
¶ True if this CDF was created in backward-compatible mode (for opening with CDF library before 3.x)
-
add_to_cache
(varname, num)[source]¶ Add a variable to the name-to-number cache
This maintains a cache of name-to-number mappings for zVariables to keep from having to query the CDF library constantly. It’s mostly an internal function.
Parameters: varname : bytes
name of the zVariable. Not this is NOT a string in Python 3!
num : int
number of the variable
-
add_attr_to_cache
(attrname, num, scope)[source]¶ Add an attribute to the name-to-number cache
This maintains a cache of name-to-number mappings for attributes to keep from having to query the CDF library constantly. It’s mostly an internal function.
Parameters: varname : bytes
name of the zVariable. Not this is NOT a string in Python 3!
num : int
number of the variable
scope : bool
True if global scope; False if variable scope.
-
attr_num
(attrname)[source]¶ Get the attribute number and scope by attribute name
This maintains a cache of name-to-number mappings for attributes to keep from having to query the CDF library constantly. It’s mostly an internal function.
Parameters: attrname : bytes
name of the zVariable. Not this is NOT a string in Python 3!
Returns: out : tuple
attribute number, scope (True for global) of this attribute
Raises: CDFError : if variable is not found
-
checksum
(new_val=None)[source]¶ Set or check the checksum status of this CDF. If checksums are enabled, the checksum will be verified every time the file is opened.
Returns: out : boolean
True if the checksum is enabled or False if disabled
Other Parameters: new_val : boolean
True to enable checksum, False to disable, or leave out to simply check.
-
clear_from_cache
(varname)[source]¶ Mark a variable deleted in the name-to-number cache
Will remove a variable, and all variables with higher numbers, from the variable cache.
Does NOT delete the variable!
This maintains a cache of name-to-number mappings for zVariables to keep from having to query the CDF library constantly. It’s mostly an internal function.
Parameters: varname : bytes
name of the zVariable. Not this is NOT a string in Python 3!
-
clear_attr_from_cache
(attrname)[source]¶ Mark an attribute deleted in the name-to-number cache
Will remove an attribute, and all attributes with higher numbers, from the attribute cache.
Does NOT delete the variable!
This maintains a cache of name-to-number mappings for attributes to keep from having to query the CDF library constantly. It’s mostly an internal function.
Parameters: attrname : bytes
name of the attribute. Not this is NOT a string in Python 3!
-
clone
(zVar, name=None, data=True)[source]¶ Clone a zVariable (from another CDF or this) into this CDF
Parameters: zVar :
Var
variable to clone
Returns: out :
Var
The newly-created zVar in this CDF
Other Parameters: name : str
Name of the new variable (default: name of the original)
data : boolean (optional)
Copy data, or only type, dimensions, variance, attributes? (default: True, copy data as well)
-
close
()[source]¶ Closes the CDF file
Although called on object destruction (
__del__()
), to ensure all data are saved, the user should explicitly callclose()
orsave()
.Raises: CDFError : if CDF library reports an error Warns: CDFWarning : if CDF library reports a warning
-
col_major
(new_col=None)[source]¶ Finds the majority of this CDF file
Returns: out : boolean
True if column-major, false if row-major
Other Parameters: new_col : boolean
Specify True to change to column-major, False to change to row major, or do not specify to check the majority rather than changing it. (default is check only)
-
compress
(comptype=None, param=None)[source]¶ Set or check the compression of this CDF
Sets compression on entire file, not per-variable.
See section 2.6 of the CDF user’s guide for more information on compression.
Returns: out : tuple
(comptype, param) currently in effect
Other Parameters: comptype : ctypes.c_long
type of compression to change to, see CDF C reference manual section 4.10. Constants for this parameter are in
const
. If not specified, will not change compression.param : ctypes.c_long
Compression parameter, see CDF CRM 4.10 and
const
. If not specified, will choose reasonable default (5 for gzip; other types have only one possible parameter.)See also
Examples
- Set file
cdffile
to gzip compression, compression level 9: >>> cdffile.compress(pycdf.const.GZIP_COMPRESSION, 9)
- Set file
-
copy
()[source]¶ Make a copy of all data and attributes in this CDF
Returns: out :
CDFCopy
SpaceData
-like object of all data
-
classmethod
from_data
(filename, sd)[source]¶ Create a new CDF file from a SpaceData object or similar
The CDF named
filename
is created, opened, filled with the contents ofsd
(including attributes), and closed.sd
should be a dictionary-like object; each key will be made into a variable name. An attribute calledattrs
, if it exists, will be made into global attributes for the CDF.Each value of
sd
should be array-like and will be used as the contents of the variable; an attribute calledattrs
, if it exists, will be made into attributes for that variable.Parameters: filename : string
name of the file to create
sd : spacepy.datamodel.SpaceData
data to put in the CDF. This structure cannot be nested, i.e., it must contain only
dmarray
and noSpacedata
objects.
-
new
(name, data=None, type=None, recVary=True, dimVarys=None, dims=None, n_elements=None, compress=None, compress_param=None)[source]¶ Create a new zVariable in this CDF
Note
Either
data
ortype
must be specified. If type is not specified, it is guessed fromdata
.Parameters: name : str
name of the new variable
Returns: out :
Var
the newly-created zVariable
Other Parameters: data
data to store in the new variable. If this has a an
attrs
attribute (e.g.,dmarray
), it will be used to populate attributes of the new variable.type : ctypes.c_long
CDF type of the variable, from
const
. See section 2.5 of the CDF user’s guide for more information on CDF data types.recVary : boolean
record variance of the variable (default True)
dimVarys : list of boolean
dimension variance of each dimension, default True for all dimensions.
dims : list of int
size of each dimension of this variable, default zero-dimensional. Note this is the dimensionality as defined by CDF, i.e., for record-varying variables it excludes the leading record dimension. See
Var
.n_elements : int
number of elements, should be 1 except for CDF_CHAR, for which it’s the length of the string.
compress : ctypes.c_long
Compression to apply to this variable, default None. See
Var.compress()
.compress_param : ctypes.c_long
Compression parameter if compression used; reasonable default is chosen. See
Var.compress()
.Raises: ValueError : if neither data nor sufficient typing information
is provided.
Notes
Any given data may be representable by a range of CDF types; if the type is not specified, pycdf will guess which the CDF types which can represent this data. This breaks down to:
- If input data is a numpy array, match the type of that array
- Proper kind (numerical, string, time)
- Proper range (stores highest and lowest number provided)
- Sufficient resolution (EPOCH16 required if datetime has microseconds or below.)
If more than one value satisfies the requirements, types are returned in preferred order:
- Type that matches precision of data first, then
- integer type before float type, then
- Smallest type first, then
- signed type first, then
- specifically-named (CDF_BYTE) vs. generically named (CDF_INT1)
So for example, EPOCH_16 is preferred over EPOCH if
data
specifies below the millisecond level (rule 1), but otherwise EPOCH is preferred (rule 2).For floats, four-byte is preferred unless eight-byte is required:
- absolute values between 0 and 3e-39
- absolute values greater than 1.7e38
This will switch to an eight-byte double in some cases where four bytes would be sufficient for IEEE 754 encoding, but where DEC formats would require eight.
-
raw_var
(name)[source]¶ Get a “raw”
Var
object.Normally a
Var
will perform translation of values for certain types (to/from Unicode for CHAR variables on Py3k, and to/from datetime for all time types). A “raw” object does not perform this translation, on read or write.This does not affect the data on disk, and in fact it is possible to maintain multiple Python objects with access to the same zVariable.
Parameters: name : str
name or number of the zVariable
-
readonly
(ro=None)[source]¶ Sets or check the readonly status of this CDF
If the CDF has been changed since opening, setting readonly mode will have no effect.
Note
Closing a CDF that has been opened readonly, or setting readonly False, may take a substantial amount of time if there are many variables in the CDF, as a (potentially large) cache needs to be cleared. Consider specifying
readonly=False
when opening the file if this is an issue. However, this may make some reading operations slower.Returns: out : Boolean
True if CDF is read-only, else False
Other Parameters: ro : Boolean
True to set the CDF readonly, False to set it read/write, or leave out to check only.
Raises: CDFError : if bad mode is set
-
save
()[source]¶ Saves the CDF file but leaves it open.
If closing the CDF,
close()
is sufficient; there is no need to callsave()
beforeclose()
.Note
Relies on an undocumented call of the CDF C library, which is also used in the Java interface.
Raises: CDFError : if CDF library reports an error Warns: CDFWarning : if CDF library reports a warning
-
var_num
(varname)[source]¶ Get the variable number of a particular variable name
This maintains a cache of name-to-number mappings for zVariables to keep from having to query the CDF library constantly. It’s mostly an internal function.
Parameters: varname : bytes
name of the zVariable. Not this is NOT a string in Python 3!
Returns: out : int
Variable number of this zvariable.
Raises: CDFError : if variable is not found