Requirements & Infrastructure¶

Hardware Requirements¶

Transfer Server¶

The transfer server is the central facility component. It hosts the Ingestor and metadata extractors.

Component	Minimum Requirements	Recommended Requirements
Memory	8 GB	16 GB or more
CPU	4 cores	8 cores or more
Network	1 Gbps	10 Gbps or more
Local Storage	80 GB	120 GB SSD or more

The transfer server must be capable of transferring large amounts of data (approximately 1–2 TB) using Globus or S3. It also runs metadata extractors that analyse hundreds of small text files. Smaller sites may use a virtual machine, while sites with substantial data-transfer requirements generally benefit from a dedicated server.

Most sites run the Ingestor and Globus on the same machine. They can run on separate machines, but both systems should share data storage.

Cache Storage¶

OpenEM can adapt to existing data-storage systems. Common setups either mount each detector's microscope storage on OpenEM service systems or copy data to central storage after acquisition. Define how data is organised and which users have access.

Cache storage should hold datasets until they are archived; a minimum retention of 30 days is typically recommended.

Component	Minimum Requirements	Recommended Requirements
Storage	50 TB	100 TB or more

Software Requirements¶

Operating System¶

Use a Linux distribution supported by Globus Connect Server, for example Red Hat Enterprise Linux and derivatives, Debian, Ubuntu, SUSE Linux Enterprise Server, or openSUSE Leap. Consult the Globus Connect Server documentation for the current supported versions.

The Ingestor software can run on Linux or Windows. Keep operating systems up to date and apply regular security updates.

Additional Packages¶

The OpenEM Standard Deployment uses Docker Compose to run the Ingestor service. Running without Docker is possible, but requires manual binary and configuration upgrades.

Network Requirements¶

Network overview

Bold lines indicate data movement; thin lines show HTTPS calls.

Firewall rules¶

Network isolation is important for the security of EM facilities. OpenEM does not require ports to be accessible from the general internet, but firewalls must permit traffic to and from trusted hosts.

Service	Port	Source	Destination	Reason
Globus	tcp/443	ingestor-server	54.237.254.192/29	Globus Control Out
Globus	tcp/443	54.237.254.192/29	ingestor-server	Globus Control In
Globus	tcp/50000-51000	ingestor-server	192.33.126.53 (lx-globus-01.psi.ch) 192.33.126.54 (lx-globus-02.psi.ch)	Globus GridFTP Out
Ingestor	tcp/443[^1]	User workstations	ingestor-server	Ingestor API
SciCat	tcp/443	User workstations	discovery.psi.ch discovery-qa.psi.ch[^2] discovery.development.psi.ch[^2]	SciCat frontend
SciCat	tcp/443	ingestor-server User workstations	dacat.psi.ch dacat-qa.psi.ch[^2] scicat.development.psi.ch[^2]	SciCat backend
SciCat	tcp/443	ingestor-server User workstations	globus-proxy.psi.ch globus-proxy.development.psi.ch[^2]	OpenEM Globus proxy

[^1]: The workstation port is configurable and independent of Globus. [^2]: URLs for testing purposes only.

Domain names¶

All OpenEM traffic is encrypted with HTTPS. Modern browsers reject session sharing and cross-origin resource sharing without valid HTTPS certificates.

Each facility should register two domains:

Globus (em-globus.facility.ch in the examples)
Ingestor (em-ingestor.facility.ch in the examples)

Usually, add both domains to the DNS server as CNAME records that resolve to the transfer server's hostname. A reverse proxy directs traffic to the correct service; see the installation documentation.