Repository.Zones.Apache.Org setup

Skip to end of metadata
Go to start of metadata

http://www.apache.org/dev/freebsd-jails.html#opie

Summary

The vm at repository.apache.org is running Ubuntu 10.04.1 LTS. Its primary purpose is to be the Maven 2 release and Snapshot repository for Apache artifacts. The artifacts are managed by Sonatype Nexus Pro (http://nexus.sonatype.org / http://www.sonatype.com/products/nexus). The release artifacts are rsynced to the central repostiory at http://repo1.maven.org/maven2. Each of these will be covered in detail below.

VM Setup

The basic setup was done but the ASF infrastructure team, that info will not be covered here. Users must have public keys available: https://svn.apache.org/repos/infra/infrastructure/trunk/ssh_keys/people/. 

In order to perform any sudo commands you need to setup an opie password

 $ opie passwd

see: http://www.apache.org/dev/freebsd-jails.html#opie

Basic setup

Add Ubunutu partner repository
Uncomment the following lines in /etc/apt/sources.list

/etc/apt/sources.list
deb http://archive.canonical.com/ubuntu lucid partner
deb-src http://archive.canonical.com/ubuntu lucid partner

Update apt-get

 $ sudo apt-get update 

Install java 1.6

 $ sudo apt-get install sun-java6-jdk 

Install curl (because I like it better then wget)

 $ sudo apt-get install curl 

Install sendmail

 $ sudo apt-get install sendmail 

Other Utilities

$ sudo apt-get install unzip

Install apache:

$ sudo apt-get install apache2
$ sudo apt-get install libapache2-mod-proxy-html
$ sudo a2enmod proxy
$ sudo a2enmod rewrite
$ sudo sudo a2enmod ssl
$ rm sites-enabled/000-default
$ ln -s /etc/apache2/mods-available/proxy_http.load /etc/apache2/mods-enabled/
/etc/apache2/sites-available/nexus

<VirtualHost _default_:80>
    DocumentRoot "/var/www"
    ProxyPreserveHost On
    ServerAdmin email@address
    ServerName repository-new.apache.org
    ErrorLog /var/log/apache2/nexus-error_log
    CustomLog /var/log/apache2/nexus-access_log common

    AddType application/x-java-jnlp-file .jnlp

    RewriteEngine On
#   RewriteLog "/var/log/apache2/nexus-rewrite_log"
#   RewriteLogLevel 3
#   allow the snapshot url to be redirected in the future if needed, when there isn't a need for the group

    #INFRA-2492
    RewriteRule ^/snapshots$ /snapshots/ [R,L]
    RewriteRule ^/snapshots/(.*) http://localhost:8081/content/groups/snapshots/$1 [P]

    #only allow gets on the snapshot group url to be http, everything else is https
    RewriteCond %{THE_REQUEST} !^(GET|HEAD)\ /content/groups/snapshots.*$
    RewriteCond %{REQUEST_URI} !^/ssl.*$ 
    RewriteRule ^(.*)$ https://repository-new.apache.org$1 [R,L]

    RewriteRule ^/content/groups/snapshots$ /content/groups/snapshots/ [R,L]
    RewriteCond %{REQUEST_URI} !^/ssl.*$ 
    RewriteRule ^(.*)$ http://localhost:8081$1 [P,L]
    SetEnv force-proxy-request-1.0 1
    SetEnv proxy-nokeepalive 1

    <Location /ssl>
      Options Indexes
      Order allow,deny
      Allow from all
    </Location>
</VirtualHost>

<IfModule mod_ssl.c>
<VirtualHost _default_:443>


    DocumentRoot "/var/www"

    ProxyRequests Off
    ProxyPreserveHost On

    <Proxy "!/ssl.cert.+">
      Order allow,deny
      Allow from all
    </Proxy>
    ProxyPass /snapshots http://localhost:8081/content/groups/snapshots

    ProxyPass / http://localhost:8081/
    <Location />
      ProxyPassReverse http:/localhost:8081/
    </Location>
	

    ServerAdmin email@address
    ServerName repository-new.apache.org
    ErrorLog /var/log/apache2/nexus-ssl-error_log
    # Possible values include: debug, info, notice, warn, error, crit,
    # alert, emerg.
   LogLevel debug 

   CustomLog /var/log/apache2/nexus-ssl-access_log common
   LogLevel debug
	#   SSL Engine Switch:
	#   Enable/Disable SSL for this virtual host.
	SSLEngine on

        SSLCertificateFile /etc/apache2/ssl/repository.apache.org.crt
        SSLCertificateKeyFile /etc/apache2/ssl/repository.apache.org.key
	
        #   Server Certificate Chain:
	#   Point SSLCertificateChainFile at a file containing the
	#   concatenation of PEM encoded CA certificates which form the
	#   certificate chain for the server certificate. Alternatively
	#   the referenced file can be the same as SSLCertificateFile
	#   when the CA certificates are directly appended to the server
	#   certificate for convinience.
	#SSLCertificateChainFile /etc/apache2/ssl.crt/server-ca.crt

	#   Certificate Authority (CA):
	#   Set the CA certificate verification path where to find CA
	#   certificates for client authentication or alternatively one
	#   huge file containing all of them (file must be PEM encoded)
	#   Note: Inside SSLCACertificatePath you need hash symlinks
	#         to point to the certificate files. Use the provided
	#         Makefile to update the hash symlinks after changes.
	#SSLCACertificatePath /etc/ssl/certs/
	#SSLCACertificateFile /etc/apache2/ssl.crt/ca-bundle.crt
        SSLCACertificateFile /etc/apache2/ssl/gd_bundle.crt

	#   Certificate Revocation Lists (CRL):
	#   Set the CA revocation path where to find CA CRLs for client
	#   authentication or alternatively one huge file containing all
	#   of them (file must be PEM encoded)
	#   Note: Inside SSLCARevocationPath you need hash symlinks
	#         to point to the certificate files. Use the provided
	#         Makefile to update the hash symlinks after changes.
	#SSLCARevocationPath /etc/apache2/ssl.crl/
	#SSLCARevocationFile /etc/apache2/ssl.crl/ca-bundle.crl

	#   Client Authentication (Type):
	#   Client certificate verification type and depth.  Types are
	#   none, optional, require and optional_no_ca.  Depth is a
	#   number which specifies how deeply to verify the certificate
	#   issuer chain before deciding the certificate is not valid.
	#SSLVerifyClient require
	#SSLVerifyDepth  10

	#   Access Control:
	#   With SSLRequire you can do per-directory access control based
	#   on arbitrary complex boolean expressions containing server
	#   variable checks and other lookup directives.  The syntax is a
	#   mixture between C and Perl.  See the mod_ssl documentation
	#   for more details.
	#<Location />
	#SSLRequire (    %{SSL_CIPHER} !~ m/^(EXP|NULL)/ \
	#            and %{SSL_CLIENT_S_DN_O} eq "Snake Oil, Ltd." \
	#            and %{SSL_CLIENT_S_DN_OU} in {"Staff", "CA", "Dev"} \
	#            and %{TIME_WDAY} >= 1 and %{TIME_WDAY} <= 5 \
	#            and %{TIME_HOUR} >= 8 and %{TIME_HOUR} <= 20       ) \
	#           or %{REMOTE_ADDR} =~ m/^192\.76\.162\.[0-9]+$/
	#</Location>

	#   SSL Engine Options:
	#   Set various options for the SSL engine.
	#   o FakeBasicAuth:
	#     Translate the client X.509 into a Basic Authorisation.  This means that
	#     the standard Auth/DBMAuth methods can be used for access control.  The
	#     user name is the `one line' version of the client's X.509 certificate.
	#     Note that no password is obtained from the user. Every entry in the user
	#     file needs this password: `xxj31ZMTZzkVA'.
	#   o ExportCertData:
	#     This exports two additional environment variables: SSL_CLIENT_CERT and
	#     SSL_SERVER_CERT. These contain the PEM-encoded certificates of the
	#     server (always existing) and the client (only existing when client
	#     authentication is used). This can be used to import the certificates
	#     into CGI scripts.
	#   o StdEnvVars:
	#     This exports the standard SSL/TLS related `SSL_*' environment variables.
	#     Per default this exportation is switched off for performance reasons,
	#     because the extraction step is an expensive operation and is usually
	#     useless for serving static content. So one usually enables the
	#     exportation for CGI and SSI requests only.
	#   o StrictRequire:
	#     This denies access when "SSLRequireSSL" or "SSLRequire" applied even
	#     under a "Satisfy any" situation, i.e. when it applies access is denied
	#     and no other module can change it.
	#   o OptRenegotiate:
	#     This enables optimized SSL connection renegotiation handling when SSL
	#     directives are used in per-directory context.
	#SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire
#	<FilesMatch "\.(cgi|shtml|phtml|php)$">
#		SSLOptions +StdEnvVars
#	</FilesMatch>
#	<Directory /usr/lib/cgi-bin>
#		SSLOptions +StdEnvVars
#	</Directory>

	#   SSL Protocol Adjustments:
	#   The safe and default but still SSL/TLS standard compliant shutdown
	#   approach is that mod_ssl sends the close notify alert but doesn't wait for
	#   the close notify alert from client. When you need a different shutdown
	#   approach you can use one of the following variables:
	#   o ssl-unclean-shutdown:
	#     This forces an unclean shutdown when the connection is closed, i.e. no
	#     SSL close notify alert is send or allowed to received.  This violates
	#     the SSL/TLS standard but is needed for some brain-dead browsers. Use
	#     this when you receive I/O errors because of the standard approach where
	#     mod_ssl sends the close notify alert.
	#   o ssl-accurate-shutdown:
	#     This forces an accurate shutdown when the connection is closed, i.e. a
	#     SSL close notify alert is send and mod_ssl waits for the close notify
	#     alert of the client. This is 100% SSL/TLS standard compliant, but in
	#     practice often causes hanging connections with brain-dead browsers. Use
	#     this only for browsers where you know that their SSL implementation
	#     works correctly.
	#   Notice: Most problems of broken clients are also related to the HTTP
	#   keep-alive facility, so you usually additionally want to disable
	#   keep-alive for those clients, too. Use variable "nokeepalive" for this.
	#   Similarly, one has to force some clients to use HTTP/1.0 to workaround
	#   their broken HTTP/1.1 implementation. Use variables "downgrade-1.0" and
	#   "force-response-1.0" for this.
	BrowserMatch "MSIE [2-6]" \
		nokeepalive ssl-unclean-shutdown \
		downgrade-1.0 force-response-1.0
	# MSIE 7 and newer should be able to use keepalive
	BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown

</VirtualHost>
</IfModule>

link it to sites-enabled

 $ sudo ln -s /etc/apache2/sites-available/nexus /etc/apache2/sites-enabled/nexus 

Enable mod-proxy

/etc/apache2/mods-enabled/proxy.conf
<IfModule mod_proxy.c>
        #turning ProxyRequests on and allowing proxying from all may allow
        #spammers to use your proxy to send email.

        ProxyRequests Off

        <Proxy *>
                AddDefaultCharset off
                Order deny,allow
                Allow from all
                #Allow from .example.com
        </Proxy>

        # Enable/disable the handling of HTTP/1.1 "Via:" headers.
        # ("Full" adds the server version; "Block" removes all outgoing Via: headers)
        # Set to one of: Off | On | Full | Block

        ProxyVia On
</IfModule>

Restart apache

 $ apache2ctl -k graceful 

Install Shorewall

$ apt-get install shorewall
$ apt-get install shorewall-doc
$ cd /etc/shorewall/
$ cp /usr/share/doc/shorewall/default-config/interfaces .
$ cp /usr/share/doc/shorewall/default-config/policy .
$ cp /usr/share/doc/shorewall/default-config/rules .
$ cp /usr/share/doc/shorewall/default-config/zones .
/etc/shorewall/interfaces
#
# Shorewall version 4 - Interfaces File
#
# For information about entries in this file, type "man shorewall-interfaces"
#
# The manpage is also online at
# http://www.shorewall.net/manpages/shorewall-interfaces.html
#
###############################################################################
#ZONE	INTERFACE	BROADCAST	OPTIONS
net     eth0            detect
/etc/shorewall/zones
#
# Shorewall version 4 - Zones File
#
# For information about this file, type "man shorewall-zones"
#
# The manpage is also online at
# http://www.shorewall.net/manpages/shorewall-zones.html
#
###############################################################################
#ZONE	TYPE		OPTIONS		IN			OUT
#					OPTIONS			OPTIONS
fw	firewall
net     ipv4
loc     ipv4
/etc/shorewall/policy
#
# Shorewall version 4 - Policy File
#
# For information about entries in this file, type "man shorewall-policy"
#
# The manpage is also online at
# http://www.shorewall.net/manpages/shorewall-policy.html
#
###############################################################################
#SOURCE	DEST	POLICY		LOG	LIMIT:		CONNLIMIT:
#				LEVEL	BURST		MASK
fw net ACCEPT
fw loc ACCEPT
net all DROP info
loc net ACCEPT info
all all REJECT info
/etc/shorewall/rules
#
# Shorewall version 4 - Rules File
#
# For information on the settings in this file, type "man shorewall-rules"
#
# The manpage is also online at
# http://www.shorewall.net/manpages/shorewall-rules.html
#
####################################################################################################################################################
#ACTION		SOURCE		DEST		PROTO	DEST	SOURCE		ORIGINAL	RATE		USER/	MARK	CONNLIMIT	TIME
#							PORT	PORT(S)		DEST		LIMIT		GROUP
#SECTION ESTABLISHED
#SECTION RELATED
SECTION NEW
ACCEPT net fw tcp 80
ACCEPT net fw tcp 443
ACCEPT net fw tcp 22
Ping/ACCEPT net $FW

Apply the rules:

 $sudo shorewall safe-restart 
Warning
Make sure you can open a new ssh session, you do NOT want to lock yourself out.

Install Nexus

Add Nexus user

$ sudo useradd --home /home/nexus --shell /bin/bash nexus
$ sudo mkdir /home/nexus
$ sudo chown nexus:nexus /home/nexus

Nexus work dir is mounted to /x1

$ sudo mkdir /x1/nexus-work
$ sudo chown nexus:nexus /x1/nexus-work
$ sudo mkdir -p /home/nexus/nexus/sonatype-work
$ sudo chown -R nexus:nexus /home/nexus/nexus
$ sudo ln -s /x1/nexus-work/ /home/nexus/nexus/sonatype-work/nexus

Install and configure nexus via script:

$ /home/nexus/nexus/upgrade-scripts/update-rao.sh <remote url for nexus tgz>
/home/nexus/nexus/upgrade-scripts/update-rao.sh
#!/bin/bash

###
# NOTE: this script was created on an older version of solaris, so if something seems odd, it is.  Updating to gnu tools is not a problem.
###

NEXUS_USER=nexus
NEXUS_GROUP=nexus
NEXUS_BUNDLE_URL=$1
NEXUS_DIR=/home/nexus/nexus
TIME_STAMP=$(date +%Y%m%d-%H%M%S)
#TIME_STAMP=20101208-025022
NEXUS_BUNDLE_TMP=$NEXUS_DIR/tmp/$TIME_STAMP
NEXUS_WORK_DIR=$NEXUS_DIR/sonatype-work/nexus
CURL_USER=remote-nexus-user
GXT_BASE_PLUGIN_URL=https://repository.sonatype.org/service/local/repositories/releases/content/org/sonatype/nexus/plugins/nexus-gxt-base-plugin/1.0.0/nexus-gxt-base-plugin-1.0.0-bundle.zip
STATS_PLUGIN_URL=https://repository.sonatype.org/service/local/repositories/sonatype-internal/content/com/sonatype/nexus/plugin/nexus-central-stat-plugin/1.0.1/nexus-central-stat-plugin-1.0.1-bundle.zip

USAGE="Usage: update-rao.sh <url-to-bundle>"

##
# Change to j2ee-repository user
##

# TODO: change user ?

##
# Check if path is valid
##
echo $NEXUS_BUNDLE_URL

# TODO: improve this by using relative path

if [ "" == "$NEXUS_BUNDLE_URL" ]; then
  echo "Missing bundle url"
  echo "$USAGE"
  exit 1
fi

##
# Change to repository folder
# cd /opt/j2ee/domains/sonatype.org/repository
##
cd "$NEXUS_DIR"

if [ ! -d "$NEXUS_BUNDLE_TMP" ]; then
  mkdir -p "$NEXUS_BUNDLE_TMP"
fi


##
# Download bundle
##

cd $NEXUS_BUNDLE_TMP 
#the root certs on this box are out of date
echo "Enter password for $CURL_USER"
curl -C - -O "$NEXUS_BUNDLE_URL" --user $CURL_USER --insecure 

BUNDLE_FILE_NAME=`echo "$NEXUS_BUNDLE_URL"| sed s_.*/__`
echo "Bundle file name: $BUNDLE_FILE_NAME"
BUNDLE_NAME=`echo "$NEXUS_BUNDLE_URL"| sed s_.*/__ | sed s/-bundle.*//`
echo "New bundle name: $BUNDLE_NAME"
NEXUS_BUNDLE_TMP_DIR=$NEXUS_BUNDLE_TMP/$BUNDLE_NAME
echo "Using temporary directory: $NEXUS_BUNDLE_TMP_DIR"


if [ -f "$NEXUS_BUNDLE_TMP/$BUNDLE_FILE_NAME" ]; then
  echo "Extracting bundle $NEXUS_BUNDLE_URL to $NEXUS_BUNDLE_TMP"
else
  echo "File not found $NEXUS_BUNDLE_URL"
  echo "$USAGE"
  exit 1
fi  

##
# Extract tar.gz
##
# echo tar -zxf "$NEXUS_BUNDLE_TMP/$BUNDLE_FILE_NAME" -C "$NEXUS_BUNDLE_TMP"
tar -zxf "$NEXUS_BUNDLE_TMP/$BUNDLE_FILE_NAME" -C "$NEXUS_BUNDLE_TMP"
##
# Update configurations
##

##
# add branding image to plexus.properties
##
echo "Updating plexus.properties"
echo "branding.image.path=/home/nexus/nexus/sonatype-work/nexus/asf_logo.png" >> "$NEXUS_BUNDLE_TMP_DIR/conf/plexus.properties"

##
# update conf/plexus.properties
# webapp-context-path=/
##
echo "Updating plexus.properties webapp-context-path"
cp "$NEXUS_BUNDLE_TMP_DIR/conf/plexus.properties" "$NEXUS_BUNDLE_TMP_DIR/conf/plexus.properties.bak"
sed s_webapp-context-path=.*_webapp-context-path=/_ "$NEXUS_BUNDLE_TMP_DIR/conf/plexus.properties" > "$NEXUS_BUNDLE_TMP_DIR/conf/plexus.properties.new"
mv "$NEXUS_BUNDLE_TMP_DIR/conf/plexus.properties.new" "$NEXUS_BUNDLE_TMP_DIR/conf/plexus.properties"

##
# Update bin/jsw/conf/wrapper.conf
# wrapper.java.command=/opt/java/sdk/1.6/bin/java
##
echo "Updating wrapper.conf"
cp "$NEXUS_BUNDLE_TMP_DIR/bin/jsw/conf/wrapper.conf" "$NEXUS_BUNDLE_TMP_DIR/bin/jsw/conf/wrapper.conf.bak"
echo "wrapper.java.additional.6=-Djavax.net.ssl.trustStore=/home/nexus/nexus/sonatype-work/nexus/keystore/ldap.keystore" >> "$NEXUS_BUNDLE_TMP_DIR/bin/jsw/conf/wrapper.conf"
echo "wrapper.java.additional.7=-Djava.net.ssl.trustStorePassword=raoapache" >> "$NEXUS_BUNDLE_TMP_DIR/bin/jsw/conf/wrapper.conf"
echo "wrapper.java.umask=0022" >> "$NEXUS_BUNDLE_TMP_DIR/bin/jsw/conf/wrapper.conf"
echo "wrapper.umask=0022" >> "$NEXUS_BUNDLE_TMP_DIR/bin/jsw/conf/wrapper.conf"

sed -f "$NEXUS_DIR/upgrade-scripts/wrapper.conf.sed" "$NEXUS_BUNDLE_TMP_DIR/bin/jsw/conf/wrapper.conf" > "$NEXUS_BUNDLE_TMP_DIR/bin/jsw/conf/wrapper.conf.new"
mv "$NEXUS_BUNDLE_TMP_DIR/bin/jsw/conf/wrapper.conf.new" "$NEXUS_BUNDLE_TMP_DIR/bin/jsw/conf/wrapper.conf"

# symlink the robots.txt
##
#ln -s /opt/j2ee/domains/sonatype.org/nexus-robots.txt "$NEXUS_BUNDLE_TMP_DIR/runtime/apps/nexus/webapp/"

##
# Update the permissions of the extracted bundle
##
chown -R $NEXUS_USER:$NEXUS_GROUP $NEXUS_BUNDLE_TMP_DIR

##
# Stop Nexus
##
/etc/init.d/nexus stop

##
# Backup previous configuration
##
cp -R $NEXUS_WORK_DIR/conf $NEXUS_WORK_DIR/conf-$TIME_STAMP

##
# Backup previous optional plugins
##
mv $NEXUS_WORK_DIR/plugin-repository $NEXUS_WORK_DIR/plugin-repository-$TIME_STAMP

##
# Create plugin-repository since we just moved it
##
mkdir -p $NEXUS_WORK_DIR/plugin-repository

##
# Put new optional plugins in place
##
# mv $NEXUS_BUNDLE_TMP_DIR/runtime/apps/nexus/optional-plugins/nexus-archetype-plugin* $NEXUS_WORK_DIR/plugin-repository/
mv $NEXUS_BUNDLE_TMP_DIR/runtime/apps/nexus/optional-plugins/nexus-branding-plugin* $NEXUS_WORK_DIR/plugin-repository/
# mv $NEXUS_BUNDLE_TMP_DIR/runtime/apps/nexus/optional-plugins/nexus-unpack-plugin* $NEXUS_WORK_DIR/plugin-repository/
# mv $NEXUS_BUNDLE_TMP_DIR/runtime/apps/nexus/optional-plugins/enterprise-crowd-plugin* $NEXUS_WORK_DIR/plugin-repository/

##
# Add the stats plugin
##

curl "$GXT_BASE_PLUGIN_URL" -o /tmp/gxt-nexus-plugin.zip
unzip /tmp/gxt-nexus-plugin.zip -d $NEXUS_WORK_DIR/plugin-repository/
rm /tmp/gxt-nexus-plugin.zip

curl "$STATS_PLUGIN_URL" -o /tmp/stats-nexus-plugin.zip --user $CURL_USER
unzip /tmp/stats-nexus-plugin.zip -d $NEXUS_WORK_DIR/plugin-repository/
rm /tmp/stats-nexus-plugin.zip

##
# Update the permissions of the plugin repository
##
chown -R $NEXUS_USER:$NEXUS_GROUP $NEXUS_WORK_DIR/plugin-repository/

##
# Move bundle into place
##
NEXUS_NEW_BUNDLE=$NEXUS_DIR/$BUNDLE_NAME-$TIME_STAMP
echo "Moving Nexus bundle in place $NEXUS_NEW_BUNDLE, before symlinking to $NEXUS_DIR/nexus"
mv "$NEXUS_BUNDLE_TMP_DIR" "$NEXUS_NEW_BUNDLE"

##
# Update symlink
##
if [ -h "$NEXUS_DIR/current" ]; then
  rm "$NEXUS_DIR/current"
fi

ln -s "$NEXUS_NEW_BUNDLE" "$NEXUS_DIR/current"
echo chown -R $NEXUS_USER:$NEXUS_GROUP "$NEXUS_DIR/current"
chown -R $NEXUS_USER:$NEXUS_GROUP "$NEXUS_DIR/current"

##
# Start Nexus
##
/etc/init.d/nexus start

/home/nexus/nexus/upgrade-scripts/wrapper.conf.sed
s_\#wrapper.java.additional.10=-XX:+HeapDumpOnOutOfMemoryError_wrapper.java.additional.8=-XX:+HeapDumpOnOutOfMemoryError_
s_wrapper.startup.timeout=300_wrapper.startup.timeout=1200_
s_wrapper.java.initmemory=.*_wrapper.java.initmemory=768_

Setup Maven Central sync

Add Maven Sync User

$ sudo useradd --home /home/central --shell /bin/bash central
$ sudo mkdir /home/central
$ sudo chown central:central /home/central

Setup the maven user key:

$ sudo su - central
$ mkdir .ssh
$ cd .ssh
$ wget http://www.ibiblio.org/maven/id_dsa.pub
$ mv id_dsa.pub authorized_keys
$ exit

Configure Backups

Configured by ASF-infra team.

Migrating from old server

"sync-from-rao"
#! /bin/bash

# split in two different commands, if not it takes WAY to long (and timesout)

/usr/bin/time rsync --verbose\
  --compress \
  --times \
  --itemize-changes \
  --stats \
  --recursive \
  --exclude-from "/home/nexus/rsync-excluded" \
  --exclude "/storage" \
  --rsync-path=/opt/sfw/bin/rsync \
  -h \
  --rsh "ssh" \
   nexus@repository.apache.org:/export/home/nexus/nexus-professional/sonatype-work/nexus/ /x1/nexus-work/

/usr/bin/time rsync --verbose\
  --compress \
  --times \
  --itemize-changes \
  --stats \
  --recursive \
  --exclude-from=/home/nexus/rsync-excluded \
  --rsync-path=/opt/sfw/bin/rsync \
  -h \
  --rsh "ssh" \
   nexus@repository.apache.org:/export/home/nexus/nexus-professional/sonatype-work/nexus/storage/ /x1/nexus-work/storage/

This is less useful, but here for completeness. These are things we have backed up over time and have not cleaned out.

rsync-excluded
conf-20101208-052817/
indexer-old/
plugin-repository-20101208-050637/
conf-back/
plugin-repository-20101208-052817/
conf-back-1.4.0/
conf.toby/
timeline-old/
orgapachewink-012/
conf-20101208-050637/

Monitor disk usage

I bet there are a hundred different ways to do this on a *nix box, I just picked the first one I googled.
/etc/cron.daily/diskAlert
#!/bin/sh
df -H | grep -vE '^Filesystem|tmpfs|cdrom|none' | awk '{ print $5 " " $1 }' | while read output;
do
  echo $output
  usep=$(echo $output | awk '{ print $1}' | cut -d'%' -f1  )
  partition=$(echo $output | awk '{ print $2 }' )
  if [ $usep -ge 90 ]; then
    echo "Running out of space \"$partition ($usep%)\" on $(hostname) as on $(date)" |
     mail -s "Alert: Almost out of disk space $usep%" email@address1 email@address2
  fi
done

Increase ulimit

Note: this has not become a problem with RAO, but has on other large instances

Default open file limit is usually 1024.

To increase this add this to /etc/security/limits.conf (where "nexus" is the UID of the user running Nexus).

/etc/security/limits.conf
 
#<domain> <type> <item> <value> 
# 
nexus hard nofile 4096 
nexus soft nofile 4096 

This will take effect immediately for new processes, so all you need to do is restart Nexus

Note: If using init.d script, make sure you are running with RUN_AS_USER set. Besides the obvious reasons, if you run as root pam authentication will not be invoked and the limits specified above won't take effect.

Note 2: On Ubuntu, you also need to add the following line to /etc/pam.d/common-session

/etc/pam.d/common-session
 
session required pam_limits.so 
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.