Getting Started With Geode
1.0 - Getting Started With Geode
Welcome to Geode, this blog entry is a beginners guide and tutorial covering the usage and features of Apache Geode. Geode is a performance orientated in-memory datastore with features that can be used for data caching, a NoSQL database, lightweight distributed messaging or for distributed compute tasks. Each of these aspects of Geode will be explored in greater detail in future blog articles with today's focus on simply getting the Geode source, building and then validating with a very simple example using Geode demonstrating basic caching of a key-value pair.
Chances are if you are reading this blog then you have already been looking at the Apache Geode website and this has further sparked your interest in what is Geode and how you can use it. Currently Geode is an incubating project and as such hasn't yet met all the requirements to be a first level Apache project, as an incubating project this provides the opportunity for new open source contributions and their developers to address Apache organization requirements, build a community and get the source into the public. Geode itself is derived from the long running commercial GemFire product and today has a rich feature set that can be immediately be used for creating applications. Let's dive in...
One assumption we are going to make upfront is that all the operations are being done within a Linux terminal window and using the Bash shell.
Next validate that a compatible version of the Java JDK is installed. The Java version needs to be a version greater than JDK 1.7.75.
$ java -version
java version "1.8.0_31"
Java(TM) SE Runtime Environment (build 1.8.0_31-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.31-b07, mixed mode)
$ jps -v
12195 Jps -Denv.class.path=. -Dapplication.home=/hostdisk/jdk1.8.0_31 -Xms8m
If you see an error or other issues using jps, you may not have a full JDK installed which will be important for later blogs as tools such as jconsole, jstack and jmap will used.
Then validate that git is installed and available which will be used to download the source code from the Apache servers.
$ git --version
git version 1.7.9.5
Now we are ready to get the Geode source and build Geode. The next step is to use git to clone the Geode source repository to your host machine.
$ git clone https://git-wip-us.apache.org/repos/asf/incubator-geode.git
Cloning into 'incubator-geode'...
remote: Counting objects: 12658, done.
remote: Compressing objects: 100% (8547/8547), done.
remote: Total 12658 (delta 4802), reused 10141 (delta 3412)
Receiving objects: 100% (12658/12658), 27.18 MiB | 1.84 MiB/s, done.
Resolving deltas: 100% (4802/4802), done.
The source code is also mirrored from the Apache server onto GitHub and can be downloaded from there as well. In most cases downloading the source should only take a few minutes at most. Once completed there should be a directory called incubator-geode created which now contains a copy of the Geode source repository.
Change into the new incubator-geode directory and list the files and directory.
$ cd incubator-geode
$ ls
build.gradle etc gemfire-core gemfire-joptsimple gemfire-junit gemfire-web-api gradle.properties gradlew.bat NOTICE RUNNING.txt COMPILING.txt gemfire-assembly gemfire-jgroups gemfire-json gemfire-web gradle gradlew LICENSE.txt README.md settings.gradle
Because the Geode project uses git, the whole project along with many feature branches are available. The initial default starting branch is the master branch which should be considered stable and is only periodically updated. Check which branch you are using with the following command.
$ git branch
* master
If on the master branch like above then change to the develop branch. The develop branch is currently very active with updates and is where new features are being added.
$ git branch develop origin/develop
$ git checkout develop
Checking out files: 100% (2822/2822), done.
Switched to branch 'develop'
The file build.gradle is the build script which is used by the Gradle build tool to create the Geode binary. You can find out more about Gradle from http://gradle.org/ but for our simple build needs we are just going to use just one command. Don't worry if you don't have Gradle installed, the script will download the required files to run the build and create the binaries. So for just getting started quickly we will skip doing the tests at this time but you should run the full test suite as some future time (especially if making changes to the Geode source).
$./gradlew clean build installDist -Dskip.tests=true
:gemfire-assembly:clean UP-TO-DATE
:gemfire-core:clean UP-TO-DATE
:gemfire-jgroups:clean UP-TO-DATE
:gemfire-joptsimple:clean UP-TO-DATE
:gemfire-json:clean UP-TO-DATE
:gemfire-junit:clean UP-TO-DATE
…
:gemfire-web-api:compileTestJava UP-TO-DATE
:gemfire-web-api:processTestResources UP-TO-DATE
:gemfire-web-api:testClasses UP-TO-DATE
:gemfire-web-api:distributedTest UP-TO-DATE
:gemfire-web-api:integrationTest UP-TO-DATE
:gemfire-web-api:test UP-TO-DATE
:gemfire-web-api:check UP-TO-DATE
:gemfire-web-api:build
BUILD SUCCESSFUL
Total time: 4 mins 5.779 secs
The build process will download Gradle if needed, then it will use the build script to create the binary files and normally will perform unit tests to validate the build. If the build took more than 5 minutes then you may have forgotten the skip.tests flag, the tests are fairly extensive and take many hours to complete. Once the build is complete, the Geode binary artifacts are placed in the directory gemfire-assembly/build/install/apache-geode and should now be ready to use. Time to change to this directory and do a few simple tests to validate the build for ourselves.
$ cd gemfire-assembly/build/install/apache-geode
$ ls
bin config lib tools
There should be four directories created, the bin directory contains scripts to run the gfsh command line tool, config contains some default configuration files, lib is all the jar files used by Geode and last is the tools directory which is where extensions to Geode live. Lets start a simple Geode distributed cache and put a little data into it to test it ourselves.
First we will start the Geode command shell and use this to start everything and enter some data. We will discuss further in a future blog using the gfsh shell, for now follow the basic commands.
$ ./bin/gfsh
_________________________ __
/ _____/ ______/ ______/ /____/ /
/ / __/ /___ /_____ / _____ /
/ /__/ / ____/ _____/ / / / /
/______/_/ /______/_/ /_/ v1.0.0-incubating-SNAPSHOT
Monitor and Manage GemFire
gfsh>
Now that the gfsh shell is started lets start the first process which is the Locator and this provides a number of services to the cache with the most important being it helps servers find each other by providing a list of members started in the distributed system. The Locator doesn't store any application data but instead is used to manage and share metadata about the servers and is a critical component which needs to be started first.
gfsh> start locator --name=locator
Starting a GemFire Locator in /hostdisk/incubator-geode/gemfire-assembly/build/install/apache-geode/locator...
Locator in /hostdisk/incubator-geode/gemfire-assembly/build/install/apache-geode/locator on localhost[10334] as locator is currently online.
Process ID: 10773
Uptime: 17 seconds
GemFire Version: 1.0.0-incubating-SNAPSHOT
Java Version: 1.8.0_31
Log File: /hostdisk/incubator-geode/gemfire-assembly/build/install/apache-geode/locator/locator.log
JVM Arguments: -Dgemfire.enable-cluster-configuration=true -Dgemfire.load-cluster-configuration-from-dir=false -Dgemfire.launcher.registerSignalHandlers=true -Djava.awt.headless=true -Dsun.rmi.dgc.server.gcInterval=9223372036854775806
Class-Path: /hostdisk/incubator-geode/gemfire-assembly/build/install/apache-geode/lib/gemfire-core-1.0.0-incubating-SNAPSHOT.jar:/hostdisk/incubator-geode/gemfire-assembly/build/install/apache-geode/lib/gemfire-core-dependencies.jar
Successfully connected to: [host=localhost, port=1099]
Cluster configuration service is up and running.
Next lets start two servers that will manage and store our user created data.
gfsh> start server --name=server1
Starting a GemFire Server in /hostdisk/incubator-geode/gemfire-assembly/build/install/apache-geode/server1...
Server in /hostdisk/incubator-geode/gemfire-assembly/build/install/apache-geode/server1 on localhost[40404] as server1 is currently online.
Process ID: 10946
Uptime: 4 seconds
GemFire Version: 1.0.0-incubating-SNAPSHOT
Java Version: 1.8.0_31
Log File: /hostdisk/incubator-geode/gemfire-assembly/build/install/apache-geode/server1/server1.log
JVM Arguments: -Dgemfire.default.locators=10.118.32.27[10334] -Dgemfire.use-cluster-configuration=true -XX:OnOutOfMemoryError=kill -KILL %p -Dgemfire.launcher.registerSignalHandlers=true -Djava.awt.headless=true -Dsun.rmi.dgc.server.gcInterval=9223372036854775806
Class-Path: /hostdisk/incubator-geode/gemfire-assembly/build/install/apache-geode/lib/gemfire-core-1.0.0-incubating-SNAPSHOT.jar:/hostdisk/incubator-geode/gemfire-assembly/build/install/apache-geode/lib/gemfire-core-dependencies.jar
The first server is started using a default port of 40404, so in starting a second server we will need to use a different server port than what the first server is using.
gfsh>start server --name=server2 --server-port=40405
Starting a GemFire Server in /hostdisk/incubator-geode/gemfire-assembly/build/install/apache-geode/server2...
Server in /hostdisk/incubator-geode/gemfire-assembly/build/install/apache-geode/server2 on localhost[40405] as server2 is currently online.
Process ID: 11081
Uptime: 4 seconds
GemFire Version: 1.0.0-incubating-SNAPSHOT
Java Version: 1.8.0_31
Log File: /hostdisk/incubator-geode/gemfire-assembly/build/install/apache-geode/server2/server2.log
JVM Arguments: -Dgemfire.default.locators=10.118.32.27[10334] -Dgemfire.use-cluster-configuration=true -XX:OnOutOfMemoryError=kill -KILL %p -Dgemfire.launcher.registerSignalHandlers=true -Djava.awt.headless=true -Dsun.rmi.dgc.server.gcInterval=9223372036854775806
Class-Path: /hostdisk/incubator-geode/gemfire-assembly/build/install/apache-geode/lib/gemfire-core-1.0.0-incubating-SNAPSHOT.jar:/hostdisk/incubator-geode/gemfire-assembly/build/install/apache-geode/lib/gemfire-core-dependencies.jar
All the members of the distributed system can be listed, some of values below will be different based on the environment but you should at this point see three members ( locator, server1 and server2).
gfsh>list members
Name | Id
------- | -------------------------------------
server1 | localhost(server1:10946)
locator | localhost(locator:10773:locator)
server2 |localhost(server2:11081)
To store data in the servers a Region must be created. The Region is very similar to a Java hashmap in usage with many additional features with one of the most important being that the data stored within the region will be shared between the two servers.
gfsh> create region --name=region --type=REPLICATE
Member | Status
------- | -------------------------------------
server2 | Region "/region" created on "server2"
server1 | Region "/region" created on "server1"
gfsh>list regions
List of regions
---------------
region
Time to put some data into our servers and use the region that was created earlier. Again similar to a hashmap a key-value pair will be inserted or put into the region to be stored by the servers. In this case the key is a simple string as well as the value.
gfsh>put --key=abc --value=def --region=region
Result : true
Key Class : java.lang.String
Key : abc
Value Class : java.lang.String
Old Value :
Once the data has been put in the servers the value can retrieved using the get command and the key.
gfsh>get --key=abc --region=region
Result : true
Key Class : java.lang.String
Key : abc
Value Class : java.lang.String
Value : def
Let's exit from gfsh.
gfsh> exit
Exiting...
From the command line we can see that gfsh has started three processes that continue to run after stopping the gfsh shell.
$ jps
10946 ServerLauncher
11081 ServerLauncher
10773 LocatorLauncher
21340 Jps
Back into gfsh and time to stop the servers.
$ ./bin/gfsh
Reconnect to the locator using the default port of 10334 after having exited and restarted gfsh.
gfsh>connect --locator=localhost[10334]
Connecting to Locator at [host=localhost, port=10334] ..
Connecting to Manager at [host=localhost, port=1099] ..
Successfully connected to: [host=localhost, port=1099]
Before stopping servers we should clean up a bit and delete the region that was created for testing the build.
gfsh>destroy region --name=region
"region" destroyed successfully.
Stop the servers and locator processes. Once both servers are stopped the data which was stored only in memory will no longer be available even if the servers are restarted.
gfsh>stop server --name=server1
Stopping Cache Server running in /hostdisk/incubator-geode/gemfire-assembly/build/install/apache-geode/server1 on localhost[40404] as server1...
Process ID: 10946
Log File: /hostdisk/incubator-geode/gemfire-assembly/build/install/apache-geode/server1/server1.log
....
gfsh>stop server --name=server2
Stopping Cache Server running in /hostdisk/incubator-geode/gemfire-assembly/build/install/apache-geode/server2 on localhost[40405] as server2...
Process ID: 11081
Log File: /hostdisk/incubator-geode/gemfire-assembly/build/install/apache-geode/server2/server2.log
....
gfsh>stop locator --name=locator
Stopping Locator running in /hostdisk/incubator-geode/gemfire-assembly/build/install/apache-geode/locator on localhost[10334] as locator...
Process ID: 10773
Log File: /hostdisk/incubator-geode/gemfire-assembly/build/install/apache-geode/locator/locator.log
...
No longer connected to localhost[1099].
gfsh>quit
Exiting...
If you inspect the build directory there have been three new directories created, one for the Locator and each Server created by gfsh and are named the same as the name parameter used in the gfsh commands to start each process. The log files for each process will be written to these directories and if you encounter any issues then those log files should be reviewed for exceptions.
At the end here we have completed getting Geode, building it from source and doing some manual testing (putting and getting a string from the cache). We have introduced the Locator and Servers along with using the GFSH shell to create and start these processes. In the next blog entry we will discuss the use cases for Geode and how it can be used in an application.
For additional details and information see the following URL´s:
https://geode-docs.cfapps.io/docs/tools_modules/gfsh/configuring_gfsh.html
https://cwiki.apache.org/confluence/display/GEODE/Index#Index-Geodein5minutes
https://geode-docs.cfapps.io/docs/tools_modules/gfsh/quick_ref_commands_by_area.html
https://geode-docs.cfapps.io/docs/basic_config/data_regions/chapter_overview.html