Print-friendly Version

Contents:

1. Overall System Architecture of Sun Fire B1600 Blade System Chassis
1.1 Introduction
1.2 System Structure
1.3 Software -- Operating System
1.4 Software -- System Service Processors (SSPs)
1.5 Hardware Availability
2. Blade Hardware Architectures
2.1 Internal I/O Devices
2.2 Midplane Connector
3 System Service Processors (SSPs)
3.1 Switch Hardware Features
3.2 Key Service Processor Features
3.3 External Connectors
4 Dual-Power Supply Units
5 General Design Requirements
5.1 Blade Identification Labeling
5.2 Human Factors/Ergonomics
5.3 Shelf/Enclosure
5.4 FRU Generic Requirements
5.5 Thermal -- Overall Power and Dissipation
5.6 Shelf Cooling Scheme
5.7 Component Cooling
5.8 Serviceability
5.9 Shelf and Midplane
5.10 Field Replaceable Units
5.11 FRU Insertion Forces
5.12 Indicators and Alarms
5.13 Air Filtration
5.14 Diagnostic Ability
5.15 Environmental Characteristics
5.16 Climatic Environments
5.17 Dynamic Environments
5.18 Safety
5.19 Regulatory Compliance
5.20 Electromagnetic Compatibility -- Emissions and Immunity
5.21 Certification Requirements for Third-Party Blade FRUs
5.22 Cable Management

1 Overall System Architecture of Sun Fire B1600 Blade System Chassis

This document describes the requirements for the Sun Fire B1600 Blade System Chassis and field-replaceable units (FRUs). The content, created by Sun Market Development Engineering, is targeted to developers who will be developing with Blade FRUs in the third-party market space. The article is a high-level discussion of issues ranging from environmental to hardware constraints. Additional documents in the Blade Computing Development Kit will cover development issues in more detail. Please send your comments to: blade-dk-help@sun.com.

1.1 Introduction

Blade systems are designed to be network centric. Network centric means that the primary means by which Blade Servers deliver services and communicate with the clients of those services is via Ethernet/IP network(s). Blade Servers rely on the network to communicate with peers and to access additional resources, such as storage and backup. Also, the system is managed through the network.

1.2 System Structure

A Blade system is a 3RU high, rack-mounting enclosure that contains up to 16 complete server units with built-in disk drive and lights-out management controller, dual redundant Layer 2 gigabit Ethernet switches, dual redundant service processors, and dual redundant power supplies. Service is single-sided for any Blade FRU and two-sided overall. All cabling is to the FRUs in the rear of the chassis or a 1U pass-through between Blade systems. Each FRU is fully enclosed to allow ruggedness and reliability in handling.

1.3 Software -- Operating System

Each server unit runs its own operating system and applications, and is essentially independent of the others with respect to its basic operation. Figure 1 shows system components and connectivity for the Blade server system.

Figure 1: Blade System Components and Connectivity

1.4 Software -- System Service Processors (SSPs)

Blades, switches, and the service processor will be running management agents. DHCP (dynamic host configuration protocol) will be used to configure the Blade networks, providing IP addresses, host name, and network resource names (DNS servers and so on). Additionally, DHCP may be used as part of the net boot necessary to install or deploy software stacks to the Blades. The switch processors will be able to run DHCP in either a master or secondary role (to a system management network resident DHCP master). Identification of a Blade to DHCP will be by hostID utilizing the clientID field in the DHCP protocol. Each Blade has a unique hostID that will be constant over replacement operations. The hostIDs provisioned for a shelf will be stored on the SSP FRU-ID device and distributed to the Blades by the system SSP in the out-of-band serial management interface to the Blade Support Chip (BSC) before the Blade runs its initial prom code. The distribution algorithm for hostIDs will be such that the hostID is bound to a slot.

The serial links provide out-of-band interfaces between each of the service processors and the Blade Support Chip and the domain console. These will be used to access the lights-out management (LOM) functionality on each Blade, to acquire environmental information, to control the fault and be ready to remove LEDs, to pass fault reporting and diagnosis information, to allow prom code to enable different boot modes for normal operation, to install, and to run diagnostics remotely.

Each Blade will run SNMP management agents delivering equipment and basic resource monitoring information -- both to management applications in the data network and via the out-of-band management system and through the SSP to the management network. In addition, the agents running on the switches will permit configuration of the network fabric.

1.5 Hardware Availability

The architectural features that contribute to the high availability of the platform are as follows:

• No single point of failure
• Highly reliable, passive midplane
• Dual, redundant power supply units
• Dual, redundant switches and Blade network interfaces
• Dual, redundant system service processors
• All FRUs are hot-swap to minimize system impact for replacement operations
• Lights-out management, including environmental monitoring, provided on all Blades
• Blades are essentially stateless, allowing rapid reconfiguration of spares from deployment server
• Blades utilize the watchdogs and SASM-based local health monitoring to operate in a fail fast mode
• Chassis with passive midplane
• Blades comprise processor, memory, disk, LOM, BSC and network interfaces

2 Blade Hardware Architectures

Figure 2: Blade Server -- Block Diagram

2.1 Internal I/O Devices

Internal I/O devices include:

• XBus real-time clock without battery
• RTC, real-time clock is programmed, having requested the date and/or time from the SSP via the BSC
• 1-Mbyte Flash PROM (OBP and POST)
• SouthBridge; only 512 Kbyte is accessible at any one time
• 2.5-inch UltraDMA100 hard disk accessed using the primary IDE interface of the SouthBridge
• BSC and FRU-ID EEPROM

Notes: A 16-Kbyte I2C EEPROM contains the BSC variables and FRU-ID. The EEPROM is nominally divided into 8 Kbyte for the FRU-ID and 8 Kbyte for the BSC variables.

• OBP EEPROM

Notes: The prom variables are stored in a second 16-Kbyte I2C EEPROM. The EEPROM contains 8 Kbyte for the OBP variables, and 8 Kbyte is unused.

• Temperature monitor for FRU ambient temperatures
• Temperature monitor for the CPU and ambient temperatures
• Blade Support Chip (BSC), which provides the following features:
  
  
  • Dual access (Blade and SSP) to FRU-ID EEPROM
  • Communication channels between Blade and system service processor
  • Control of the active, service-required and ready-to-remove LEDs
  • Field-upgradable firmware
  • Implements a watchdog function for Blade FRU
  • Monitors the speed of the CPU fan
  • Communicates with Blade FRU processors by the XBus
  • Power sequencing and reset
    
    
• DCDC converters
• Powered from two, diode or'ed, 12V supplies
• Dual 12V input secondary converters

2.2 Midplane Connector

The Blade connects to the midplane through a 40-pin SCA-2 connector. (Figure 3 shows data for the midplane connector.)

Figure 3: Midplane Connector

3 System Service Processors (SSPs)

The system service processors are composed of the following:

1. A 16-port switch used by the Blade FRU for in-band communication
2. A service processor
3. External connections

In other, related documents, when we talk about SSP, we will refer to all three functional subsystems as one unit.

3.1 Switch Hardware Features

• 16 x 1-Gbit SERDES links to each Blade, with each link consisting of a TX +/- and RX+/- signals
  
  
• 8 x 10/100/1000BaseT connections available at RJ45 connectors for connection to the outside world
  
  
• Switch interconnected through the 10-Gbit XGMII ports
  
  
• Control processor
  
  
• Console UART providing a console connection from the switch to the SSP, which can be accessed via the SSP
  
  
• 10/100 MAC/PHY providing a management connection to the switch from the outside world via a three-port hub; this link is fixed at 100 Mbit/sec
  
  
• Watchdog restarts the switch
  
  
• Three-port 10/100 hub provides a common connection to the management network for the switch
  
  

3.2 Key Service Processor Features

• Octal UARTs, the TXD and RXD signals are used: These are buffered to allow hot-swap of the SSP and the Blades.
  
  
• The RXD lines are pulled up so that when a Blade is not present or not powered the RXD signals appear negated. The RXD and TXD signals are LVTTL levels and are pulled up to 3.3 V.
  
  
• The CD_L input for each port is used to detect the presence of the corresponding Blade.
  
  
• DUART provides serial connections to the rear panel and a console connection to the switch processor.
  
  

3.3 External Connectors

Figures 4 and 5 contain data and the layout for external connectors. RJ45 - 10/100/1000BaseT Data Network Connectors are arranged as a 4x2 array. These RJ45 connectors provide the connection to the switch. Each connector has integral green Link_Present and Link_Active indicators. Note that the Link_Present indicator is always on the left regardless of the orientation of the RJ45 connector.

Figure 4: External Connectors - Data Network Connector

RJ45 - RS232 SSP Console Connection

This RJ45 connector provides access to the SSP console. This connector should be clearly differentiated from the Ethernet Management Connection described in the preceding section. Figure 5 shows the RS232 SSP Console Connection.

Figure 5: External Connectors -- RS232 SSP Console Connection

4 Dual Power Supply Units

The power system is made up of power supply units (PSUs) that convert AC input power to 12 VDC and DC-to-DC converters that change the 12 VDC to the lower voltages required within the system FRUs.

Redundancy in the 12 VDC supply is achieved by having two 1050-watt output PSUs with independent AC inputs. Either PSU is capable of supplying all the power needs of a complete Blade system from a single line cord.

In addition to the redundant PSUs, the power system has other design features to achieve high system availability. Each PSU has its 12-volt source divided into five independent outputs, with each output having its own current limiting circuit. If a system fault causes an over-current on one of the PSU outputs, it results in an under-voltage condition on that output only. The other four outputs continue to supply power to the system. An output may remain in an over-current condition for an indefinite length of time without damaging the PSU. When an over-current fault is cleared, the PSU output will automatically reset to its normal operating mode.

The 10 12-VDC PSU outputs (five from each PSU) are used to form a 25-position output load array for the Blade FRUs as shown in the following table.

Table 2: B0-B15 = Blades, PSU0d-PSU1d = PSU Diagnostics/Fans

A fault at any position in the output load array can cause an under-voltage on one output of each PSU. However, only the fault position will have both 12VDC inputs in an under-voltage condition. All other positions in the output load array will have at least one 12VDC input active and will remain in operation.

5 General Design Requirements

All enclosures should act as a shield for EMC and ESD immunity, and should incorporate the minimum amount of grounding fingers or gasketing where applicable. Dissimilar metallic materials should be avoided wherever possible, to prevent problems arising from galvanic action (potential difference between materials). Continuous folded sheet metal construction will be used where practicable to reduce potential EMI leakage through gaps. Where necessary, metal parts will be bonded to reduce EMI radiation. Provision should be made for a chassis/frame ground point and ESD wrist strap ground point on the rear of the enclosure.

5.1 Blade Identification Labeling

The front bezel should provide a free surface area for custom labeling. Each label should indicate the type of FRU and functionality.

5.2 Human Factors and Ergonomics

The product design should minimize the number and the use of standard and special tools for maintenance and installation. LED/indicator architecture should be based on the latest Sun Service Indicator Standard Guidelines and Service Indicator. The FRUs should have indicators conforming to Table 3.

Table 3: Enclosure and FRU Indicators

Note: There are two active/green LEDs, one each for 'AC power good' and 'DC power good.'

5.3 Shelf/Enclosure

The enclosure and mechanical components will be unique for the Blade product, and should comprise a metalwork enclosure with apertures at front and rear to accommodate the FRUs. A vertically mounted midplane arrangement will allow for the electrical interconnection of the FRUs. The enclosure should incorporate guide and keying systems for easy insertion of the FRUs into the enclosure and for accurate alignment and mating of the interface connectors between the FRUs and midplane. There should not be any cabling within the shelf between the different modules.

5.4 FRU Generic Requirements

Each Blade FRU should be:

• Hot swappable
• Positively retained in the shelf by its own latch or locking mechanism
• Keyed and guided to prevent incorrect positioning and insertion into the enclosure
• Self-contained in its own robust enclosure to facilitate EMC, ESD, handling, storage, and transportation
• A 'sealed' unit with no field- or customer-serviceable parts

Also, the number of Blades depends on system configuration. Each Blade should incorporate a fascia and handle mechanism.

5.5 Thermal -- Overall Power and Dissipation

• It is anticipated that each Blade will dissipate between 35 and 45 W.
• The expected average dissipation of the Blade shelf is about 850 W.
• The heat release of the system will be considerable and will be highlighted in relevant approvals and/or compliance reports, but will not form a pass/fail criterion in itself.

5.6 Shelf Cooling Scheme

The shelf will be cooled by forced air cooling principles, where intake of air is from the front of the shelf, and out via exhaust outlets at the rear of the shelf. DC axial fans will provide the necessary bulk airflow and will be located at the rear of the shelf.

The main shelf cooling will be achieved by drawing ambient air in through the Blade fascia. This air exhausts at the rear of the Blades and is then drawn through the midplane openings and finally into the power supply units. The Blades should be designed such that they are self-cooling, and the shelf should be able to operate and self-cool in the event of the failure of either power supply.

Where a Blade System shelf is not fully configured with the maximum number of Blades, dummy or blanking modules should be used to ensure that the correct cooling characteristics are maintained across the shelf. The dummy or blanking Blades should emulate the look and feel of a fully populated Blade.

5.7 Component Cooling

Each Blade FRU should carry its own cooler/blower unit. Other "hot" components should be suitably cooled where deemed necessary. Temperature sensors should monitor each FRU enclosure, and should warn the user when hazardous thermal conditions arise. The async communication of the BSC should be used to monitor temperature.

5.8 Serviceability

Service is a single-sided operation for any given Blade FRU.

5.9 Shelf and Midplane

The optimum position for the installed system is in the lower half of the customer's rack, which will allow for easy removal and replacement of the Blade FRUs.

5.10 Field Replaceable Units

Customer-replaceable Blade FRUs should be removable without the use of tools, from the enclosure.

5.11 FRU Insertion Forces

The theoretical insertion force for all FRUs is determined by the type of midplane interface connector and by any mechanical forces present in each of the FRU latching/retention mechanisms. Theoretical insertion forces for the Blade FRUs are ~ 4.0 kgf maximum.

5.12 Indicators and Alarms

All LED service indicators should be as stated in the section titled "Human Factors and Ergonomics."

5.13 Air Filtration

No provision is made for air filtration.

5.14 Diagnostic Ability

For the majority of faults, the Blade system is a relatively simple system in which to determine which FRU is faulty. This is because each FRU has very few external connections and most of these connections only go to the SSPs or the power supplies.

5.15 Environmental Characteristics

The Blade System is classified as a rack-mounting server, destined for rack-mounting applications only, in climatically controlled server rooms or environments. A summary of the operating environmental conditions is provided as follows:

• Operating temperature: 5 C - 35 C
• Operating humidity: 10% - 90%
• Operating altitude: 3000 m

Note: Blade FRU is not required to be certified through NEBS.

5.16 Climatic Environmental Characteristics

The operational specification for Blade Systems should be maintained over the full range of climatic conditions defined by the preceding specification, including the following parameters:

• Temperature (operating and non-operating)
• Humidity (operating and non-operating)
• Altitude (operating and non-operating)

Third-party Blade FRUs should not cause a violation of operational specifications.

5.17 Dynamic Environments

The operational specification for Blade FRUs should be maintained over the full range of dynamic conditions defined by the preceding specification, including the following parameters:

• Vibration (operating and non-operating)
• Shock (operating and non-operating)

5.18 Safety

The safety requirements of Blade systems are defined in this section. Certification from the following regulatory programs is required:

• CSA
• TUV "GS Mark"
• IEC "CB Scheme"

5.19 Regulatory Compliance

The product will be submitted to the relevant qualification approval programs necessary to allow delivery into world-wide markets. Certification from regulatory agencies will be obtained when necessary to satisfy legal requirements.

5.20 Electromagnetic Compatibility -- Emissions and Immunity

Blade systems should meet the electromagnetic emissions and immunity requirements as defined in the preceding specification, where the following parameters are applicable:

• Radiated Emissions (Electric Field)
• Conducted Emissions
• Harmonics Emissions AC
• Voltage Flicker AC
• Conducted Immunity
• ESD Immunity
• Electric Field Immunity
• Surge Immunity
• Electrical Fast Transients
• Voltage Dips and Interruptions
• Sun Voltage Margin
• Sun Frequency Margin
• Sun Customer On/Off
• Safety of Institute of Transportation Engineers (ITE)

5.21 Certification Requirements for Third-Party Blade FRUs

Certification from the following regulatory programs is to be obtained:

• EU "CE Mark" (Europe)
• FCC (United States)
• DOC (Canada)
• VCCI (Japan)
• GOST (Russia)
• BSMI (Taiwan)
• ACA "C-Tick" (Australia and New Zealand)

Table 4: Certification Requirements

5.22 Cable Management

Shelf cable management is required. The use of standard I/O connectors with locking mechanisms should be used wherever possible. Contact Sun Microsystems for recommended cable management (at blade-dk-help@sun.com).

April 2003