Posted on

Saving some variables from a netCDF to a new file

This post was originally published here

The NCO (netCDF Operator) command ncks (netCDF Kitchen Sink).

From the documentation:

The nickname “kitchen sink” is a catch-all because ncks combines most features of ncdump and nccopy with extra features to extract, hyperslab, multi-slab, sub-set, and translate into one versatile utility. ncks extracts (a subset of the) data from input-file and and writes (or pastes) it in netCDF format to output-file, and optionally writes it in flat binary format to binary-file, and optionally prints it to screen.

/…/

ncks extracts (and optionally creates a new netCDF file comprised of) only selected variables from the input file (similar to the old ncextr specification). Only variables and coordinates may be specifically included or excluded—all global attributes and any attribute associated with an extracted variable are copied to the screen and/or output netCDF file.

The flag for extracting variables is -v (followed by variable name(s) separated by commas):

ncks -v var1,var2 in.nc out.nc no space after the comma!

In case you’ve forgotten what the names of your variables are, do:

ncdump -h in__filename_.nc

-h prints headers only (and not the values). I usually direct the output of ncdump to a text file:

ncdump -h in__filename_.nc > ncdump.txt

Also, if you forgot some of the variables that you wanted then you don’t have to do the whole list again – NCO is always willing to append variables. So if you run:

ncks -v var3 in.nc out.nc

but the out.nc already exists, then NCO will prompt you with this:

ncks: out.nc exists—e'xit,o’verwrite (i.e., delete existing file), or `a’ppend (i.e., replace duplicate variables in and add new variables to existing file) (e/o/a)?

So you can enter a and hit ‘return’.

Posted on

My bash aliases

This post was originally published here

If you find yourself using some commands always with the same flags, then it would make sense to define them as alieses, by putting them into your .bashrc file like this (log out and back in for it to take effect):

-a for ls shows hidden files (files that start with a dot, like .bashrc) and -l displays more information than just the file/folder names (permissions for example).

_–color=auto _colours folders, executables and symbolic links.

-a for qstat displays more information.

Both -m and -M for qsub mean messages. For -m:

b – Mail is sent at the beginning of the job.

e – Mail is sent at the end of the job.

a – Mail is sent when the job is aborted or rescheduled.

And -M is the flag before the email address(es).

The last one (I call it disk) displays the sizes of one level of subfolders and orders them too (correct ordering is done by the really cool -h option, as apposed to the numeric sort -n, which would think that 1.4GB>1.4TB).

Posted on

Add up two variables of a netCDF file

This post was originally published here

NCO:ncap2 is the function to do it:

ncap2 -s 'new_var=var1+var2' in_filename.nc out_filename.nc

The output file will have all of the variables that exist in the input file as well as the new_var. Add -O if your input and output files are the same (overwrite).

I do not know what the -s stands for.

BUT the new_var will have the same long_name as the first variable used for summing (i.e. it could make some things a bit confusing). To change it, use a very complicated (but allegedly also very powerful) NCO:ncatted. Fortunately, its documentation has just the right example:

Change the value of the long_name attribute for variable T from whatever it currently is to “temperature”:

ncatted -a long_name,T,o,c,temperature in.nc