Emu¶
- Emu (the bird)
- Emus are curious birds who are known to follow and watch other animals and humans. Emus do not sleep continuously at night but in several short stints sitting down. [..]. (Wikipedia).
Emu is a Python package with some test proccess for Web Processing Services (WPS). Currently it is using the PyWPS 4.x server.
Full documentation is available on ReadTheDocs or in the docs directory.
Installation¶
Install from Conda¶
Install the emu
Conda package:
$ conda install -c birdhouse -c conda-forge emu
$ emu --help
Install from GitHub¶
Check out code from the Emu GitHub repo and start the installation:
$ git clone https://github.com/bird-house/emu.git
$ cd emu
$ conda env create -f environment.yml
$ source activate emu
$ python setup.py develop
… or do it the lazy way¶
The previous installation instructions assume you have Anaconda installed.
We provide also a Makefile
to run this installation without additional steps:
$ git clone https://github.com/bird-house/emu.git
$ cd emu
$ make clean # cleans up a previous Conda environment
$ make install # installs Conda if necessary and runs the above installation steps
Start Emu PyWPS service¶
After successful installation you can start the service using the emu
command-line.
$ emu start --help # show help
$ emu start # start service with default configuration
OR
$ emu start --daemon # start service as daemon
loading configuration
forked process id: 42
The deployed WPS service is by default available on:
http://localhost:5000/wps?service=WPS&version=1.0.0&request=GetCapabilities.
Note
Remember the process ID (PID) so you can stop the service with kill PID
.
You can find which process uses a given port using the following command (here for port 5000):
$ netstat -nlp | grep :5000
Check the log files for errors:
$ tail -f pywps.log
… or do it the lazy way¶
You can also use the Makefile
to start and stop the service:
$ make start
$ make status
$ tail -f pywps.log
$ make stop
Use Ansible to deploy Emu on your System¶
Use the Ansible playbook for PyWPS to deploy Emu on your system.
Configuration¶
Command-line options¶
You can overwrite the default PyWPS configuration by using command-line options. See the Emu help which options are available:
$ emu start --help
--hostname HOSTNAME hostname in PyWPS configuration.
--port PORT port in PyWPS configuration.
Start service with different hostname and port:
$ emu start --hostname localhost --port 5001
Use a custom configuration file¶
You can overwrite the default PyWPS configuration by providing your own
PyWPS configuration file (just modifiy the options you want to change).
Use one of the existing sample-*.cfg
files as example and copy them to etc/custom.cfg
.
For example change the hostname (demo.org) and logging level:
$ cd emu
$ vim etc/custom.cfg
$ cat etc/custom.cfg
[server]
url = http://demo.org:5000/wps
outputurl = http://demo.org:5000/outputs
[logging]
level = DEBUG
Start the service with your custom configuration:
# start the service with this configuration
$ emu start -c etc/custom.cfg
Developer Guide¶
Building the docs¶
First install dependencies for the documentation:
$ make bootstrap_dev
$ make docs
Running tests¶
Run tests using pytest.
First activate the emu
Conda environment and install pytest
.
$ cd emu
$ source activate emu
$ conda install pytest flake8 # if not already installed
Run quick tests (skip slow and online):
$ pytest -m 'not slow and not online'"
Run all tests:
$ pytest
Check pep8:
$ flake8
Run tests the lazy way¶
Do the same as above using the Makefile
.
$ make test
$ make testall
$ make pep8
Bump a new version¶
Make a new version of Emu in the following steps:
- Make sure everything is commit to GitHub.
- Update
CHANGES.rst
with the next version. - Dry Run:
bumpversion --dry-run --verbose --new-version 0.8.1 patch
- Do it:
bumpversion --new-version 0.8.1 patch
- … or:
bumpversion --new-version 0.9.0 minor
- Push it:
git push
- Push tag:
git push --tags
See the bumpversion documentation for details.
Tutorials¶
Tutorial: using Docker¶
Emu WPS is available as docker image. You can download the docker image from DockerHub or build it from the provided Dockerfile.
Start the container with the following command:
$ docker run -d -p 5000:5000 --name=emu birdhouse/emu
It is using the port 5000 for the PyWPS service and also to access the WPS outputs.
You can map the container port also to another port on your machine, for example: -p 8094:5000
(your machine port=8094, container port=5000).
Check the docker logs:
$ docker logs emu
Show running docker containers:
$ docker ps
Run a GetCapabilites WPS request:
Run DescribeProcess WPS request for Hello:
Execute Hello process with you user name:
Install Birdy WPS command line tool from Conda:
$ conda install -c birdhouse birdhouse-birdy
Use Birdy to access Emu WPS service:
$ export WPS_SERVICE=http://localhost:5000/wps
$ birdy -h
$ birdy hello -h
$ birdy hello --name Pingu
Stop and remove docker container:
$ docker stop emu
Using docker-compose¶
Use docker-compose (you need a recent version > 1.7) to start the container:
$ git clone https://github.com/bird-house/emu.git
$ cd emu
$ docker-compose up -d
$ docker-compose logs emu
Execute tail
command in the running container to see the logs:
$ docker ps # get the container name
NAMES
emu_emu_1
$ docker exec -it emu_emu_1 tail -f /opt/wps/pywps.log
You can customize the docker-compose.yml
file.
See the docker-compose documentation.
Stop the container with:
$ docker-compose down
Build image using docker-compose¶
You can build locally a new docker image from the Dockerfile by running docker-compose:
$ docker-compose build
Tutorial: using postgres database¶
You can use a postgres database for PyWPS, the default is sqlite. PyWPS is using SQLAlchemy, see the PYWPS documentation for details.
First run the Emu default installation:
$ git clone https://github.com/bird-house/emu.git
$ cd emu
$ make clean install
The default installation is using sqlite. We now need a postgres database. If you don’t have one yet you can use a postgres docker container.
$ docker pull postgres
$ docker run --name postgres -e POSTGRES_PASSWORD=postgres -p 5432:5432 -d postgres
The postgres database is now available on default port 5432.
SQLAlchemy needs the psycopg2 postgres adapter.
You can install it via Conda into the emu
environment.
$ conda install -n emu psycopg2
The SQLAlchemy connection string for this database is:
# postgresql+psycopg2://user:password@host:port/dbname
postgresql+psycopg2://postgres:postgres@localhost:5432/postgres
Configure this connection string in etc/custom.cfg
, logging
section, database
option:
$ vim etc/custom.cfg
[logging]
level = INFO
database = postgresql+psycopg2://postgres:postgres@localhost:5432/postgres
Start the emu service:
$ emu start -c etc/custom.cfg
Your Emu WPS service should be available at the following URL:
$ firefox http://localhost:5000/wps?request=GetCapabilities&service=WPS
Processes¶
Say Hello¶
Sleep¶
-
class
emu.processes.wps_sleep.
Sleep
[source]¶ sleep Sleep Process (v1.0)
Testing a long running process, in the sleep. This process will sleep for a given delay or 10 seconds if not a valid value.
Parameters: delay (float) – Delay between every update Returns: sleep_output – Sleep Output Return type: string References
Wordcounter¶
-
class
emu.processes.wps_wordcounter.
WordCounter
[source]¶ wordcounter Word Counter (v1.0)
Counts words in a given text.
Parameters: text (text/plain) – URL pointing to a text document, for example “Alice in Wonderland”: http://www.gutenberg.org/cache/epub/19033/pg19033.txt Returns: output – Word counter result Return type: application/json References
Counts ocurrences of all words in a document.
Chomsky¶
-
class
emu.processes.wps_chomsky.
Chomsky
[source]¶ chomsky Chomsky text generator (v1.0)
Generates a random chomsky text
Parameters: times (integer) – Generates a random chomsky text. Returns: output – Chomsky text Return type: text/plain Generates a random chomsky text: http://code.activestate.com/recipes/440546-chomsky-random-text-generator/
CHOMSKY is an aid to writing linguistic papers in the style of the great master. It is based on selected phrases taken from actual books and articles written by Noam Chomsky. Upon request, it assembles the phrases in the elegant stylistic patterns that Chomsky is noted for. To generate n sentences of linguistic wisdom, type:
- (CHOMSKY n) – for example
- (CHOMSKY 5) generates half a screen of linguistic truth.
NCMeta¶
-
class
emu.processes.wps_ncmeta.
NCMeta
[source]¶ ncmeta Return NetCDF Metadata (v4)
Return metadata from a netCDF dataset, either on file or an OpenDAP service.
Parameters: - dataset (application/x-netcdf, optional) – http://test.opendap.org:80/opendap/netcdf/examples/sresa1b_ncar_ccsm3_0_run1_200001.nc.nc4
- dataset_opendap (application/x-ogc-dods, optional) – http://test.opendap.org:80/opendap/netcdf/examples/sresa1b_ncar_ccsm3_0_run1_200001.nc
Returns: output – Metadata
Return type: text/plain
References
Returns metadata of a NetCDF file or OpenDAP resource.
\ Sort by:\ best rated\ newest\ oldest\
\\
Add a comment\ (markup):
\``code``
, \ code blocks:::
and an indented block after blank line