Globus Connect Server Installation

What Globus Does

Globus is a service for reliably moving large datasets between storage systems. It authenticates the systems, starts and monitors transfers, and can resume a transfer after an interruption. The research data moves directly from the facility to PSI; Globus does not temporarily store it in a cloud service.

OpenEM uses Globus Connect Server (GCS) to make one directory on the facility's transfer server available for transfers to PSI.

The GCS configuration consists of three resources:

• An endpoint represents the complete GCS installation. A single-server installation has one data transfer node.
• A storage gateway defines who may access the attached storage and how a Globus identity is mapped to a storage account.
• A mapped collection exposes a specific directory through Globus. In this guide, it exposes only the OpenEM transfer directory.

The setup below creates a single-node Globus Connect Server 5.4 endpoint with one POSIX storage gateway and one private mapped collection. It uses only GCS basic features; a Globus subscription and guest collections are not required.

Note

Facilities using another transfer mechanism, such as the ETHZ Archiving Service, should follow that service's documentation instead.

Before You Begin

This guide assumes:

• one transfer server running a supported Ubuntu or Debian release;
• administrator access through sudo;
• a POSIX filesystem mounted on that server;
• a stable, externally routable IP address, or correctly configured NAT; and
• a terminal session on the transfer server.

GCS requires at least 8 GB RAM, synchronized system time, a Unicode locale, and the network access described below. Check the current list of supported operating systems and all requirements in the official GCS prerequisites. The OpenEM-specific hardware recommendations are listed under infrastructure requirements.

Information to Obtain First

Collect the following information before starting. Do not guess identity names or UUIDs.

Value	Obtain it from	Example
Endpoint name	Choose a descriptive name	FACILITY OpenEM Production Endpoint
Organization	Your institution	Example University
Endpoint-owner identity	Globus Web App, as described below	admin@example.org
Contact email	Your operations team	openem-support@example.org
Gateway name	Choose a descriptive name	FACILITY OpenEM Production Gateway
PSI transfer client ID	SciCat Support	00000000-0000-0000-0000-000000000000
POSIX account	Your system administrator	openem-globus
Collection path	Your storage administrator	/srv/openem/transfer
Collection name	Choose a descriptive name	FACILITY OpenEM Production Collection

Find the Endpoint-Owner Identity

The endpoint owner is the person or team account that administers this Globus endpoint. Sign in to the Globus Web App using the intended institutional account. Open Settings, select Account, and copy the complete username shown under Identity.

Copy the complete value highlighted in red, including the identity domain after the @ character.

The identity username may contain an institution-specific identifier and may not be the same as the account's email address. It must already have been used to log in to Globus. See the Globus identity FAQ for more information.

Important

The endpoint-owner identity and the PSI transfer client ID are different:

• The endpoint-owner identity belongs to the administrator creating the facility endpoint.
• The PSI transfer client ID belongs to the PSI application that later reads data from the collection.

Check Network Access

The default GCS firewall policy requires TCP 443 and TCP 50000-51000 in both directions. TCP 443 carries management, authentication, and GridFTP control traffic. TCP 50000-51000 carries the dataset directly between endpoints.

An OpenEM source endpoint is commonly restricted to PSI. Agree the exact rules with the facility network team and SciCat Support:

Port	Minimum access for a PSI-only source endpoint	Purpose
TCP 443 inbound	Current Globus Transfer service ranges	GridFTP control traffic
TCP 443 outbound	Globus and package repositories	Installation, configuration, and GCS APIs
TCP 50000-51000 outbound	Current PSI data transfer nodes	Dataset transfer to PSI

Ask SciCat Support for the current PSI data transfer node addresses. Globus documents its current service ranges and the limitations caused by tighter firewall rules in the guide to restricted firewall policies.

Warning

Blocking inbound TCP 50000-51000 prevents the facility endpoint from acting as the destination of a regular server-to-server transfer. Use the default Globus policy if the endpoint must support transfers other than the OpenEM source-to-PSI workflow.

GCS installs and configures Apache on TCP port 443. Check whether that port is already occupied:

sudo ss --tcp --listening --numeric --processes 'sport = :443'

No output means that no process is currently listening on the port. If the command lists another web server, stop here and follow the official network-use guidance before running GCS node setup.

The commands below define values as shell variables immediately before they are needed. Edit each export block before running it, and keep the same terminal open throughout the installation so previously defined variables remain available.

1. Install Globus Connect Server

Create a working directory and enter it. GCS will later write the endpoint's deployment key here.

mkdir -p globus-setup
cd globus-setup

Add the official Globus package repository and install GCS:

curl -LOs https://downloads.globus.org/globus-connect-server/stable/installers/repo/deb/globus-repo_latest_all.deb
sudo dpkg -i globus-repo_latest_all.deb
sudo apt update
sudo apt install globus-connect-server54

Confirm that the command is installed:

globus-connect-server --version

The command should print a version number. Installing the package alone does not create or activate an endpoint.

For a Linux distribution other than Ubuntu or Debian, use the commands in the official GCS installation guide, then continue with the next step.

2. Create the Endpoint

The endpoint represents the facility's GCS installation in Globus. Run this step only once for a new endpoint. First, enter the endpoint information collected under Information to Obtain First:

export ENDPOINT_NAME="FACILITY OpenEM Production Endpoint"
export ORGANIZATION="Example University"
export OWNER_IDENTITY="admin@example.org"
export CONTACT_EMAIL="openem-support@example.org"

Check the values before creating the endpoint:

printf 'Endpoint: %s\nOrganization: %s\nOwner: %s\nContact: %s\n' \
  "$ENDPOINT_NAME" "$ORGANIZATION" "$OWNER_IDENTITY" "$CONTACT_EMAIL"

Correct a value by editing and running its export command again. Then create the endpoint:

globus-connect-server endpoint setup "$ENDPOINT_NAME" \
  --organization "$ORGANIZATION" \
  --owner "$OWNER_IDENTITY" \
  --contact-email "$CONTACT_EMAIL"

The command displays a URL. Open it in a browser and authenticate with the same Globus account that contains OWNER_IDENTITY. Follow the prompts, including the Let's Encrypt terms for the automatically managed TLS certificate.

When setup finishes, the output contains the new endpoint ID and GCS domain name. Record both; they are needed when registering the endpoint with PSI.

The command also creates deployment-key.json in the current directory. This file allows a server to join and operate the endpoint. Globus cannot recover it. Protect it like a password and keep a secure backup:

chmod 600 deployment-key.json
ls -l deployment-key.json

The displayed permissions should begin with -rw-------. Never commit this file to Git or place it in the collection directory. For background information, see the official endpoint setup reference.

Important

The endpoint ID identifies the new facility endpoint. It is not PSI_TRANSFER_CLIENT_ID and must not be used in the identity mapping below.

3. Configure and Start the Node

A node is the physical or virtual server running the GCS transfer services. The following command reads deployment-key.json, configures Apache and the GCS services, and starts them through systemd:

sudo globus-connect-server node setup

This command must be run from the directory containing deployment-key.json. For a server behind NAT, first read the official NAT instructions; node setup may need the public address through --ip-address.

When the command completes without an error, continue with the administrator login. The next step verifies that Globus knows this node and reports it as active. If setup fails, recheck TCP 443, public DNS or NAT, and system time.

4. Log In to the Local GCS Manager

The remaining commands change the endpoint configuration and therefore require an administrator login:

globus-connect-server login localhost

Open the displayed URL and log in with the endpoint-owner account. The command stores a local authentication token for the GCS command-line tool; it does not create a Linux login account.

Verify the endpoint configuration:

globus-connect-server endpoint show
globus-connect-server node list

Confirm that the endpoint name is correct and the node status is active.

5. Prepare the POSIX Account and Directory

The POSIX connector accesses files with the UID and GID of a POSIX account on the data transfer node. The endpoint owner and facility users do not need such an account unless they also require data access through this collection.

Set the account name and the absolute path to the directory that Globus should expose. Adapt both values to the facility's system:

export POSIX_ACCOUNT="openem-globus"
export COLLECTION_PATH="/srv/openem/transfer"

Review the values:

printf 'POSIX account: %s\nCollection path: %s\n' \
  "$POSIX_ACCOUNT" "$COLLECTION_PATH"

Check whether the selected account already exists:

getent passwd "$POSIX_ACCOUNT"

If the command prints an account entry, use that account and do not run useradd. If it prints nothing, create a dedicated system account without a home directory or interactive login shell:

sudo useradd --system --no-create-home --shell /usr/sbin/nologin "$POSIX_ACCOUNT"

The collection directory must already contain, or receive, the datasets intended for PSI. Unless the facility automates this, operators must copy or move datasets into this directory manually.

Confirm that the path exists:

sudo test -d "$COLLECTION_PATH" && echo "Collection directory exists"

If there is no output, ask the storage administrator to create or mount the directory. Do not create it blindly if the path is expected to be a network mount; otherwise, data might be written to the transfer server's local disk.

Grant POSIX_ACCOUNT read and directory-traversal permissions using the facility's normal user, group, or ACL policy. There is no universal permission command because ownership and mounted storage differ between facilities. Do not grant this account access to unrelated storage.

Test the actual permissions as the selected account:

sudo -u "$POSIX_ACCOUNT" find "$COLLECTION_PATH" \
  -maxdepth 2 -mindepth 1 -print

The command should list the accessible files and directories without Permission denied. If the directory is currently empty, place a non-sensitive test file in it and run the check again.

The use of a POSIX account is required by this connector: GCS performs every file operation as the mapped account. Other storage connectors can use different credential types. See the official POSIX storage gateway description.

6. Create the Identity Mapping

When the PSI transfer application connects, Globus presents its identity as:

PSI_TRANSFER_CLIENT_ID@clients.auth.globus.org

Set the client ID supplied by SciCat Support. Do not use the facility endpoint ID here:

export PSI_TRANSFER_CLIENT_ID="00000000-0000-0000-0000-000000000000"

Confirm that the value is the real UUID. Do not continue while it contains the all-zero example:

printf 'PSI transfer client ID: %s\n' "$PSI_TRANSFER_CLIENT_ID"

The identity mapping translates that Globus identity to POSIX_ACCOUNT. The following command creates the mapping file from the variables defined above:

umask 077
cat > identity-mapping.json <<EOF
{
  "DATA_TYPE": "expression_identity_mapping#1.0.0",
  "mappings": [
    {
      "source": "{username}",
      "match": "${PSI_TRANSFER_CLIENT_ID}@clients.auth.globus.org",
      "literal": true,
      "output": "${POSIX_ACCOUNT}"
    }
  ]
}
EOF

"literal": true requires an exact identity match. This prevents similarly named identities from being mapped accidentally.

Review the generated file:

cat identity-mapping.json
ls -l identity-mapping.json

Verify that it contains the real PSI transfer client ID and the intended POSIX account. Its permissions should begin with -rw-------. The mapping contains no client secret, but it is still security-relevant configuration. See the official application-credential guide and Identity Mapping Guide for details.

7. Create the POSIX Storage Gateway

The storage gateway combines the POSIX connector with the authentication and mapping rules. This command allows only Globus application identities, applies the mapping file, and allows only the resulting POSIX_ACCOUNT. First, set the descriptive gateway name:

export GATEWAY_NAME="FACILITY OpenEM Production Gateway"

Review the name:

printf 'Gateway: %s\n' "$GATEWAY_NAME"

Then create the storage gateway:

globus-connect-server storage-gateway create posix \
  "$GATEWAY_NAME" \
  --domain clients.auth.globus.org \
  --identity-mapping file:identity-mapping.json \
  --user-allow "$POSIX_ACCOUNT"

A successful command prints Storage Gateway ID: followed by a UUID. Copy only that UUID into the next command:

export STORAGE_GATEWAY_ID="PASTE-STORAGE-GATEWAY-UUID-HERE"

Confirm that the variable contains the real UUID, then inspect the stored configuration:

printf 'Storage Gateway ID: %s\n' "$STORAGE_GATEWAY_ID"
globus-connect-server storage-gateway show "$STORAGE_GATEWAY_ID" \
  --include-private-policies \
  --format json

Check the displayed identity mapping, allowed domain, and allowed POSIX account. The full set of options is documented in the POSIX storage gateway reference.

8. Create the Mapped Collection

The mapped collection is the directory that Globus can access. The following command connects it to the storage gateway and makes it private. Guest collections are explicitly disabled. First, set the name shown for the collection in Globus:

export COLLECTION_NAME="FACILITY OpenEM Production Collection"

Review the name:

printf 'Collection: %s\n' "$COLLECTION_NAME"

Then create the collection:

globus-connect-server collection create \
  "$STORAGE_GATEWAY_ID" \
  "$COLLECTION_PATH" \
  "$COLLECTION_NAME" \
  --private \
  --no-allow-guest-collections

A successful command prints Collection ID: followed by a UUID. Record it:

export COLLECTION_ID="PASTE-COLLECTION-UUID-HERE"

The collection base path becomes / from the perspective of Globus. A file at $COLLECTION_PATH/example.dat therefore appears in this collection as /example.dat. Files outside COLLECTION_PATH are not exposed by this collection.

Verify the collection:

printf 'Collection ID: %s\n' "$COLLECTION_ID"
globus-connect-server collection show "$COLLECTION_ID"
globus-connect-server collection list

Confirm the collection name, storage gateway ID, base path, private status, and disabled guest collections. See the official collection create reference for all available options.

9. Record the Result

At this point, the installation has produced several different identifiers. Do not substitute one for another.

Identifier	Meaning	Where it came from
Endpoint-owner identity	Globus identity of the administrator	Globus account settings
PSI transfer client ID	Globus identity of the PSI transfer application	SciCat Support
Endpoint ID	Facility's complete GCS endpoint	endpoint setup
Storage gateway ID	POSIX access and identity-mapping configuration	storage-gateway create
Collection ID	Directory exposed for OpenEM transfers	collection create

Store the IDs and names in the facility's password manager or operations documentation. Keep deployment-key.json in a protected secret store; unlike the UUIDs, it is confidential.

10. Register and Test the Endpoint

The PSI Globus Proxy must know the new endpoint before OpenEM can request a transfer. Send the following information to SciCat Support:

• facility name;
• endpoint ID;
• GCS domain name;
• mapped collection ID; and
• collection base path.

SciCat Support will provide or confirm the facility-specific Ingestor configuration. After registration:

1. Put a small, non-sensitive test dataset in COLLECTION_PATH.
2. Request its transfer through the normal OpenEM workflow.
3. Confirm that the transfer completes and the dataset arrives at PSI.
4. Confirm that paths outside COLLECTION_PATH cannot be accessed.

For diagnostics, run:

sudo globus-connect-server self-diagnostic

The command collects GCS configuration, service status, and network checks. Do not post its output publicly; provide it to Globus or SciCat Support only when requested. The official troubleshooting guide describes common connection, permission, and certificate problems.