Blog

This post was originally published here

Firstly, start off your table in http://www.tablesgenerator.com/.

Tables Generator will do a lot for you. Its most useful features are importing from .csv and merging cells. The Booktabs table style (alternative to default table style from the menu) looks a bit nicer and is “publication quality”. Note that publication quality tables should not contain vertical lines.

Screen shot of Tables Generator

Code#1

Code #1 is the code from Tables Generator with the addition of caption, label and Latex document begin-end (so it’s compilable). Continuing from that table, let’s centre the contents of columns 1-3 and the whole table in your document, by adding centering and changing the table specs from l’s to c’s: Code #2.

Code#2

Finally, if your cell contents are long and need wrapping:

Note that if your table is too wide for your document margins, then LaTex issues a warning, not an error. So you need check for warnings like “Overfull hbox (9.75735pt too wide) in paragraph at lines 55–63” in your compilation log. A quick solution to wide cells is like this (Code#4):

Code#4

But this solution does not include decent central alignment. Using m (so m{2cm} instead of p{2cm}) would do the vertical centering (e.g. look how the first row is alligned), but still not horizontal. So following this StackOverflow post, I started defining column types and widths using the array package. See Code#5.

Code#5

Next time I might write a post on how to add extra space between lines.

This post was originally published here

This is a short note based on this.

Answer in short: Because different formulas are used to calculate the R-squared of a linear regression, depending on whether it has an intercept or not.

R2 for a linear model that has an intercept:

,

where y is the variable that the linear model is trying to predict (the response variable), y^ is the predicted value and y- is the mean value of the response variable.

And the R2 for a linear model that is forced through the origin:

,

basically the mean value of the response variable is removed from the equation, making the denominator bigger (and the result of the division smaller). The reason why the mean can not be used for this calculation is that it does not make sense any more – forcing the fit through zero kind of means adding an infinite number of (0,0) points into the dataset.

This means that the R-squared values of two different linear models (one with an intercept, one without) can not really be compared, because when the intercept is quite small compared to the residuals (basically the numerator) then the R2 “advantange” that the through-origin regression gets is relatively bigger than the decrease in residuals, when including the intercept.

This post was originally published here

convert -rotate 270 -density 300 -compress lzw in.pdf out.pdf

This post was originally published here

I don’t know if it’s good or bad, but I like when the files I’m working with are in the working directory (so instead of using pathnames to my files I can just type filename or ./filename). But to avoid copying data and wasting space, symbolic links are the way to go. The command for that is:

ln -s target_file sym_link,

where -s stands for “symbolic” (just ln would create a hard link)

However, if you are not a complete UNIX guru, then trying to access your linked files is likely to produce one of these errors:

No such file or directory OR Too many levels of symbolic links

The solution to both of these is to always use full paths to the files and their symbolic links (ln -s /home/folder/file.txt /home/folder2/file.txt). For further information, see this and this. Apparently you can have 32 levels of symbolic links, so getting a “Too many levels of symbolic links” after just creating one, means that there is some serious recursion going on.

Remove symbolic links just as you remove files:

rm sym_link

This post was originally published here

If you’ve defined alias ls='ls -al –color=auto', but want to use ls without the extra information and colouring then use

ls

or equivalently

command ls

This post was originally published here

The NCO (netCDF Operator) command ncks (netCDF Kitchen Sink).

From the documentation:

The nickname “kitchen sink” is a catch-all because ncks combines most features of ncdump and nccopy with extra features to extract, hyperslab, multi-slab, sub-set, and translate into one versatile utility. ncks extracts (a subset of the) data from input-file and and writes (or pastes) it in netCDF format to output-file, and optionally writes it in flat binary format to binary-file, and optionally prints it to screen.

/…/

ncks extracts (and optionally creates a new netCDF file comprised of) only selected variables from the input file (similar to the old ncextr specification). Only variables and coordinates may be specifically included or excluded—all global attributes and any attribute associated with an extracted variable are copied to the screen and/or output netCDF file.

The flag for extracting variables is -v (followed by variable name(s) separated by commas):

ncks -v var1,var2 in.nc out.nc no space after the comma!

In case you’ve forgotten what the names of your variables are, do:

ncdump -h in__filename_.nc

-h prints headers only (and not the values). I usually direct the output of ncdump to a text file:

ncdump -h in__filename_.nc > ncdump.txt

Also, if you forgot some of the variables that you wanted then you don’t have to do the whole list again – NCO is always willing to append variables. So if you run:

ncks -v var3 in.nc out.nc

but the out.nc already exists, then NCO will prompt you with this:

ncks: out.nc exists—e'xit,o’verwrite (i.e., delete existing file), or `a’ppend (i.e., replace duplicate variables in and add new variables to existing file) (e/o/a)?

So you can enter a and hit ‘return’.

This post was originally published here

If you find yourself using some commands always with the same flags, then it would make sense to define them as alieses, by putting them into your .bashrc file like this (log out and back in for it to take effect):

-a for ls shows hidden files (files that start with a dot, like .bashrc) and -l displays more information than just the file/folder names (permissions for example).

_–color=auto _colours folders, executables and symbolic links.

-a for qstat displays more information.

Both -m and -M for qsub mean messages. For -m:

b – Mail is sent at the beginning of the job.

e – Mail is sent at the end of the job.

a – Mail is sent when the job is aborted or rescheduled.

And -M is the flag before the email address(es).

The last one (I call it disk) displays the sizes of one level of subfolders and orders them too (correct ordering is done by the really cool -h option, as apposed to the numeric sort -n, which would think that 1.4GB>1.4TB).

This post was originally published here

NCO:ncap2 is the function to do it:

ncap2 -s 'new_var=var1+var2' in_filename.nc out_filename.nc

The output file will have all of the variables that exist in the input file as well as the new_var. Add -O if your input and output files are the same (overwrite).

I do not know what the -s stands for.

BUT the new_var will have the same long_name as the first variable used for summing (i.e. it could make some things a bit confusing). To change it, use a very complicated (but allegedly also very powerful) NCO:ncatted. Fortunately, its documentation has just the right example:

Change the value of the long_name attribute for variable T from whatever it currently is to “temperature”:

ncatted -a long_name,T,o,c,temperature in.nc

This post was originally published here

NCO:ncap2 and .total

ncap2 -s 'summed_variable=variable_to_sum.total($lat,$lon)' in.nc out.nc

Make sure to use single quotes.

If your in.nc==out.cnc then adding -A will save you from having to specify “overwrite” (see this).

ncap2 -A -s 'summed_variable=variable_to_sum.total($lat,$lon)' in.nc out.nc