Tuesday, February 6, 2018

Install WRF

After days of frustration, I finally get WRF running in my MacBook. Thanks to the help from wrfhelp@ucar.edu
$ brew install NetCDF
🍺  /usr/local/Cellar/netcdf/4.6.0: 84 files, 6MB
$ brew install mpich
🍺  /usr/local/Cellar/mpich/3.2.1_1: 887 files, 14.2MB
$ brew install zlib
Warning: zlib 1.2.11 is already installed
$ brew install libpng
Warning: libpng 1.6.34 is already installed
$ brew install jasper
Warning: jasper 2.0.14 is already installed
However, NetCDF 4.6.0 doesn’t work for WRF. I don’t know the exact reason. One possible reason is After 4.2 (2012), the Fortran netCDF library has been split off into an independent source distribution, intended to be built as a separate library, after the C library is built and installed. This separation simplifies the building and use of the C and Fortran netCDF libraries and allows them to evolve independently.
However, even if you install netCDF-Fortran 4.4.4, WRF still doesn’t work. My workable solution is based on netCDF 4.1.3, downloaded from http://www2.mmm.ucar.edu/wrf/OnLineTutorial/compilation_tutorial.php#STEP2
The tricky part has something to do with the compatibility with Mac OS high Sierra. You have to add #include <config.h>to first line of load.c file.
mkdir Build_WRF
cd ~/Downloads/Build_WRF
export DIR=~/Downloads/Build_WRF/LIBRARIES
export CC=gcc
export CXX=g++
export FC=gfortran
export FCFLAGS=-m64
export F77=gfortran
export FFLAGS=-m64
# NetCDF
cd netcdf-4.1.3
subl ncgen3/load.c
# add "#include <config.h>" to fist line
./configure --prefix=$DIR/netcdf --disable-dap \
     --disable-netcdf-4 --disable-shared
make install  
export PATH=$DIR/netcdf/bin:$PATH
export NETCDF=$DIR/netcdf

Library compatibility test

cd /Downloads/WRFV3/TESTS/Fortran_C_NETCDF_MPI_tests
cp ${NETCDF}/include/netcdf.inc .
gfortran -c 01_fortran+c+netcdf_f.f
gcc -c 01_fortran+c+netcdf_c.c
gfortran 01_fortran+c+netcdf_f.o 01_fortran+c+netcdf_c.o \
     -L${NETCDF}/lib -lnetcdff -lnetcdf
# SUCCESS test 1 fortran + c + netcdf
mpif90 -c 02_fortran+c+netcdf+mpi_f.f
mpicc -c 02_fortran+c+netcdf+mpi_c.c
mpif90 02_fortran+c+netcdf+mpi_f.o \
02_fortran+c+netcdf+mpi_c.o \
     -L${NETCDF}/lib -lnetcdff -lnetcdf
mpirun ./a.out
# SUCCESS test 2 fortran + c + netcdf + mpi

compile WRF

cd ~/Downloads/WRFV3
checking for perl5... no
checking for perl... found /usr/bin/perl (perl)
** WARNING: No path to NETCDF and environment variable NETCDF not set.
** would you like me to try to fix? [y]
$ y
Enter full path to NetCDF include directory on your system
$ /Downloads/Build_WRF/LIBRARIES/netcdf/include
Please select from among the following Darwin ARCH options:
Enter selection [1-36] : 15
Compile for nesting? (0=no nesting, 1=basic, 2=preset moves, 3=vortex following) [default 0]: 1

$ ./compile em_b_wave >& compile.log
$ subl compile.log
# Executables successfully built

other things learned

default make install path is at /usr/local/bin
Brew install path is at /usr/local/Cellar/
Port install path is at /usr/opt/
see recent changes: ls -lrt
brew install on Centos: http://linuxbrew.sh/
install pip on Centos: sudo yum install -y python-pip
check yum-installed package location: rpm -ql package-name

make install latest netcdf-c

cd /Users/jiang/Downloads/netcdf-c-4.6.0
./configure --prefix=/usr/local
make check
make install

make install lastest netcdf-fortran

cd /Users/jiang/Downloads/netcdf-fortran-4.4.4
./configure --prefix=${NFDIR}

Tricks for netcdf on high serria

\1. I did see that it said there was a problem building the executables, but typically there are errors listed further up in the file to help us to determine why the executables weren’t built. We look for errors with a capital ‘E’ (i.e., Error). All others listed are not relevant. However, again, if you don’t have netcdf built correctly, WRF will not build either.
\2. Have you tried to use our compilation tutorial web page to get the libraries installed correctly on your system?
This helps you to verify that everything is okay in your environment, and then to get the libraries installed correctly, and then test that they are compatible with each other, and with your compilers. We recommend following it word-for-word. Even if you have already built some of the libraries, it is recommended to start fresh in a clean directory and rebuild them as indicated in the page, in order to ensure a greater chance of a successful WRF build. If you follow these directions and are unable to get netcdf installed, then you will need to talk to your systems administrator to see what the problem is. We are aware of a netcdf installation problem on Mac OS High Sierra (which is why I asked), if you are using the Clang version of gfortran. I will paste below what another user sent us to solve this problem. However, beyond this problem, it will have to be solved by your systems administrator at your institute. Here is the High Sierra solution. We have not actually tested this here because we aren’t able to repeat the problem on our systems, but in case it helps you…
Some cursory googling suggests that the issue is in MacOS’s default clang compiler: https://www.unidata.ucar.edu/software/netcdf/docs/known_problems.html#clang-ncgen3
I’ve found two ways to work around this, one is a band-aid and I think the other is an actual fix. The band-aid is the one suggested at that link: you add the line
#include <config.h>
above line 8 of ncgen3/load.c. This lets clang compile correctly. However, it leaves open the question of why the heck clang is being used at all, given that we told it to use gcc.
Well, it turns out that even if you’ve used homebrew to install the latest gcc, as I have, the installed gcc-7 and g++-7 don’t get automatically symlinked to gcc and g++ the way it automatically symlinks gfortran-7 to gfortran. Thus, if you set CC=gcc, the XCode-installed /usr/bin/gcc is called, and it apparently just points back to clang. I should have caught this by checking the versions of all the compilers, but I only did gfortran —version since that’s the compiler I’ve had the most configuration issues with, and I glossed over the possibility that XCode was still first in line for C and C++.
So, I’ve had success with two solutions/workarounds. If the user doesn’t want to override XCode for some reason, they can specify CC=gcc-7 and CXX=g++-7, and that allows netcdf to compile (I haven’t tested farther than that, because I prefer the second solution). If they want to use the newer GCC for everything, they can just go to /usr/local/bin and symlink gcc-7 to gcc and g++-7 to g++, and if their path is set up correctly these should then be first priority. I’m currently in the process of going through the full compilation process to see if any new issues crop up, but this should actually help with stability given that the first time through I was compiling with gfortran 7.3 but gcc 4.whatever_Apple_gives_me. Obviously this only applies to folks that use Homebrew but should be similar for macports folks.