MiServer: Managed Linux: Service Overview

Overview

The MiServer Managed Service offers customers a managed operating system, including installation and support for the operating system, patch management, antivirus, monitoring, and backups. MiServer Managed (sometimes referred to as Managed OS) is an offering in the cloud service that leverages components from Infrastructure as a Service (IaaS) to provide service to customers. More information on components listed in IaaS can be found on the IaaS and Cloud service definitions.

For detailed information about these services refer to the Service Definition Documents and Service Level Expectation document.

Scope

The scope of this document is to provide customers more detailed information regarding components of the service.

In Scope

  • Managed OS service includes the following: OS provisioning, patch management and backups/restores.
  • Operating Systems offered to customer for new requests:
    • RHEL 6 64 Bit
  • Support Operating Systems when migrated into the service:
    • RHEL 6 64 Bit
    • RHEL 5 64 Bit

Out of Scope

  • Any system with PCI or HIPAA data
  • High Performance Computing (HPC)
  • Application support
  • Any operating system not specified in scope for the service

Expectations

  • Customers should have the capability and necessary skillsets to manage applications.
  • Customers will put servers in maintenance mode prior to performing any maintenance orperforming any work that may be disruptive on the server.
  • Applications will not be installed or supported by the Managed OS support.
  • Managed OS support will only be for OS support level.

Monitoring

The MiServer Managed service proactivity monitors the Operation System to help identify potential problems, maintain high availability, and provide prompt break\fix response times if OS issues arise. Customers are asked to put servers in maintenance mode prior to performing any maintenance or disruptive work on the server. The monitoring component in the MiServer Managed service includes however not limited to:

  • Unplanned Server Restarts\Crashes
  • Operating System & Operating System Core Services
  • MiServer Components (Antivirus, Backups, Patches, etc)
  • Service Infrastructure Components

In the event an incident is identified by the monitoring agent or through other channels the MiServer Managed team will work with the customer to take corrective measures as necessary. Please remember it’s crucial to put servers subscribed to the service in maintenance mode prior to taking any services offline intentionally or performing any disruptive maintenance. This process is to help identify servers that do not need to be responded to immediately and prevent services from being recovered when the original intent to have those services remain offline.

Nagios is the monitoring tool of choice and the MiServer team has several sample scripts available for adding customer applications or ports into monitoring as desired. Create an incident and an administrator will contact you to work on this.

For off hours support of the OS, a 24/7 contact number is required for the customer. Many common OS issues may be triggered by the application, such as excessive paging. While the MiServer team will attempt to mitigate the issue, customer approval to recycle an application, add disk space, or reboot a server may be required.

All servers will be monitored 24x7 for the following:

  • Disk & Inode usage on /, /boot, /home, /opt, /tmp, /usr/local, /var
  • Swap utilization
  • SSH local & remote access
  • Ping up / down

How to put Servers in Maintenance Mode

Log into the Nagios dashboard at http://nagios.dsc.umich.edu and either click on the Hosts link or search for your server. Under the Host Commands menu, schedule appropriate downtime.

Patch Management

It is best practice to apply security patches as soon as possible to not further risk a server from being compromised due to known exploits. All servers subscribed to the MiServer Managed service will be patched every 3 months during the MiServer infrastructure maintenance window. Additional patches may be required if critical security fixes are identified. Customers will have the option to opt out of or reschedule a specific patch date by contacting the OS team. We encourage non-production servers to be patched at least 1 week before the all server window.

Backups

Scheduled backups will include all files except remote file systems such as AFS or NFS fall into the following exclusions shown below. Customers have the ability to restore individual files on demand using the backup client installed on the server. In the event a customer requires or would like additional assistance, a service request should be filed with the MiServer Managed Server team. If a complete system restore is required, customers should work with the MiServer Managed team by requesting a system state restore.

User Management

Components of the OS managed servers will be managed under group policies. All members of the MCommunity group used to register for the service will be granted access to the server. A management script will run twice daily to grant new members of the group access to the server. Due to data security concerns, we will not automatically delete users when removed from the group. Please submit a ticket if a user no longer should have access. Additional groups can be added to the authorization file /usr/local/etc/mcomm-group.

OS Upgrades

As new Operating Systems become available the MiServer Managed service will review, evaluate and test the new Operating System. Minor releases (ie, 6.3 -> 6.4) will be upgraded as part of the standard patching window and are generally available shortly after release. Major releases will take longer to be available for MiServer, but should be fully supported within 6 months of general availability. After the new OS has been completely reviewed and fully tested within the service to reflect best practices, the new OS will become available for new provisions. RHEL does not currently support in-place upgrades for major releases. If a customer wishes to migrate to a server running the new OS, the customer will be given a new server provisioned with the new OS for 3 months. During this time the customer will have the opportunity to install/configure his or her application and migrate data to the new server.

Filesystem Configuration & Application Installs

Your new MiServer has a default 50 GB disk 0, partitioned into several filesystems. 20 GB is in the root volume group, where we keep about 7 GB unused to add space to whatever filesystems you need. This will enable the OS team to troubleshoot certain monitoring incidents after hours without calling the customer. The other 30 GB will be in a /usr/local/miserver filesystem. Additional disk requested via the portal will be used to grow the /usr/local/miserver filesystem. While not required, this filesystem is intended for application specific data / binaries / logs / etc. If requested the MiServer team can use the additional disk to create or grow any filesystem needed.

Root Level Access & System Configuration

If your application does not require root access, we recommend the creation of an “Application ID”. This ID can be used to install and own an application, with shared access granted to as many users as needed. Customers can submit an incident or create the ID(s) themselves as they desire. If root access is required, the use of sudo is required. The MiServer team can work with the customer to determine what sudo commands are appropriate. Changing the root password can cause delays if support is needed and potentially require the system be booted into init 1 for the OS team to repair a damaged system.

User Accounts & Passwords

System account passwords (root, etc) should not be changed. User accounts should be added only in accordance with the Standard Practices Guide.

SSH Access Configuration

Systems are delivered with remote root level control granted to a number of MiServer administrative hosts via host keys. No remote root access should be granted to any other hosts. Currently the administrative hosts are:

  • alethe.dsc.umich.edu
  • booby.dsc.umich.edu
  • grassbird.dsc.umich.edu
  • mandarin.dsc.umich.edu

This access is required to support your system. This list may change over time, please check this document for the current list.

User-level access via public keys, local password & kerberos are all enabled by default. Specific users can be blocked from SSH connections by adding their ID to /etc/ssh/sshd.deny. Available options include turning on RSA two-factor authentication and disabling public keys.

DenyHosts is running by default. This service will lock unauthorized IP addresses out of the system if too many failed SSH connections are detected. Please put an incident in if you are locked out. Short term mitigation is to connect from a different IP address.

IPTABLES

Please do not make any rules that would block ports 22, 111, 2049, 5666, 5667, 7001-7011

AFS

AFS is not enabled by default but can be added if requested. In your request, please specify if local home directories should be replaced by AFS.

NFS

NFS access to University-supplied value storage will be enabled on request. Other NFS access will be granted on a case by case basis.

Additional Packages

The following repositories are enabled by default. The MiServer team can install packages or a customer with appropriate sudo privileges can use yum to install packages.

  • epel6-x86_64
  • rhel-x86_64-server-6
  • rhel-x86_64-server-optional-6
  • rhn-tools-rhel-x86_64-server-6

DNS/Hostname Updates

The MiServer OS team tracks and monitors servers based on DNS values. If you want to change your server name or domain, please put in a request and we will do it for you. This will allow us to ensure that your server continues to be monitored and patched without interruption in service.

Vulnerability Scanning

ITS is providing monthly credentialed vulnerability scans. These scans will examine managed servers both locally and remotely and alert administrators to any security concerns. The local scan does require elevated access. To ensure security, the scanner will connect as an unprivileged ID via keypair access from the scanning server. It will then switch user to a user with sudo privileges. This ID is a local account only and not allowed remote access to your server.

Last Updated: 
Monday, February 20, 2017