September 18, 2019

Alfresco, JVM tuning and monitoring with JMX

Tuning and deeper inspection of the Java Virtual Machine is a very important activity for java-based applications. In an enterprise-class installation of Alfresco where often one or multiple clustered instances are running, a huge amounts of data are stored with a lot of user access that cause often a http overload. Like many other Java applications, Alfresco can be optimized by a careful and meticulous tuning of the hardware resources, specifically cpu and ram used by the JVM. I am not versed in the details regarding java thread management, you can found further details here. To make a succesfull JVM tuning, the Alfresco official wiki suggests the following approach.

1. Throw as much RAM as possible at the JVM (-Xmx32G);
2. Set the perm gen to 256M (-XX:MaxPermSize:256m);
3. Don’t add any other configuration settings.

Once the heap memory has the maximum allowed value, you can proceed with additional configuration parameters by editing the JAVA_OPTS variable which is located in the Alfresco startup script.Here the JAVA_OPTS settings in a default Alfresco installation.

JAVA_OPTS="-XX:MaxPermSize=512m -Xms128m -Xmx1024m -XX:-DisableExplicitGC"
JAVA_OPTS="${JAVA_OPTS} -Djava.awt.headless=true"
JAVA_OPTS="${JAVA_OPTS} -Dalfresco.home=/opt/alfresco-4.1.1.3"
JAVA_OPTS="${JAVA_OPTS} -Dcom.sun.management.jmxremote -Dsun.security.ssl.allowUnsafeRenegotiation=true"

In a recent documentation, Alfresco has released a table of possible values ​​for the heap memory based on the number of users accessing the system.

 jvm-tuning-00

Note that for these metrics, N concurrent users is considered equivalent to 10xN casual users that the server could support. Follow a best practice for an Alfresco installation where about 5 milions of documents are stored and  frequently solr searches are performed.

– Guest Alfresco = 4.0.0 Enterprise (2 cluster nodes)
– Guest RAM = 10 GB
– Guest CPU = 4 virtual cores
– Hypervisor = Esxi 5.0
– Host CPU = Intel Xeon E5640 2.67GHz
– Storage = 1 TB SAN (via NFS)

JAVA_OPTS="-XX:MaxPermSize=512m -Xms8000m -Xmx8000m -XX:-DisableExplicitGC"
JAVA_OPTS="${JAVA_OPTS} -Djava.awt.headless=true"
JAVA_OPTS="${JAVA_OPTS} -Dalfresco.home=/opt/alfresco-4.1.1.3"
JAVA_OPTS="${JAVA_OPTS} -Dcom.sun.management.jmxremote -Dsun.security.ssl.allowUnsafeRenegotiation=true"
JAVA_OPTS="${JAVA_OPTS} -Xss1024K -XX:NewSize=1G -XX:+UseConcMarkSweepGC -server"
JAVA_OPTS="${JAVA_OPTS} -XX:+CMSIncrementalMode -XX:CMSInitiatingOccupancyFraction=80"
JAVA_OPTS="${JAVA_OPTS} -XX:ParallelGCThreads=4 -XX:+UseParNewGC"

Now, let’s see how to perform a deep monitoring of the JVM.

Case 1. Using jstad and jvisualvm client.

– Server JVM

$ vi JAVA_HOME_SERVER/bin/jstatd.policy

grant codebase "file:${java.home}/../lib/tools.jar" {
        permission java.security.AllPermission;
};

$  ./jstatd -J-Djava.security.policy=./jstatd.policy -J-Djava.rmi.server.hostname=192.168.71.64

 

– Client JVM

$ JAVA_HOME/jvisualvm

 jvm-tuning-01

Case 2. Using JMX and jvisualvm client (Alfresco Enterprise 3.2 and higher).

– Server JVM

By default the JMX interface allows you to access Alfresco Enterprise via a standard JMX console that supports JMX Remoting like jvisualvm. There are no more settings to do for Alfresco server. Only a important note for Redhat/Centos operative systems: Alfresco has an in process RMI registry that will be started on the IP specificed by the command line property -Djava.rmi.server.hostname and the port in alfresco.rmi.services.host. If you have multiple network interfaces, you must start Alfresco defining the “java.rmi.server.hostname” with the server IP address. A second solution is to add a new local dns entry in the /etc/hosts file.

$ vi ALFRESCO_HOME/tomcat/scripts/ctl.sh
JAVA_OPTS="-Djava.rmi.server.hostname=192.168.1.23 -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv6Addresses=false"
$ vi /etc/hosts
192.168.1.23	myalfresco

 

– Client JVM

Run jvisualvm and attach a JMX connection using the default Alfresco credentials.

service:jmx:rmi:///jndi/rmi://192.168.1.23:50500/alfresco/jmxrmi
-username (default): controlRole
-password (default): change_asap

 jvm-tuning-03

 

Related posts

Leave a Reply

Your email address will not be published.