The Leading Source for Global News and Information Covering the Ecosystem of High Productivity Computing
November 24, 2006
The National Center for Data Mining at UIC has won the HPC Bandwidth Challenge at SC06 in Tampa, FL, sponsored by Qwest. Nine institutions participated in the competition. NCDM won by sustaining a data transfer rate of 8 Gb/s over a 10 Gb/s link, with a peak rate of 9.18 Gb/s during the competition window. NCDM uses its own open source software products, UDT and SECTOR, to transfer large datasets efficiently at high speeds on optical networks.
This year the Bandwidth Challenge focused on a specific facet of networking: End-to-End achievement. Competitors were asked to fully utilize one 10 Gig path, end-to-end, disk-to-disk, from SC06 in Tampa back to their home institution, using the actual production network back home. Participants were required to realize, demonstrate and publish all the configuration, troubleshooting, tuning and policies used. The SC06 show floor was connected with the major U.S. research networks, specifically: Abilene, ESnet, NLR PacketNet, NLR FrameNet, and HOPI. The U.S. research networks provided transit for the international networks with which they peer.
NCDM transferred Sloan Digital Sky Survey Data (SDSS) between the SC06 show floor in Tampa, Fl and its lab in 4223 SEL at the University of Illinois in Chicago. It used SECTOR, the newly developed distributed data space management system. SECTOR transparently manages the file locating and data moving, while the NCDM developed UDT software is used for the actual data transfer. The data transfer was disk to disk over one 10Gb/s shared routed link between SC06 and UIC, via StarLight.
SDSS is systematically mapping a quarter of the entire sky to produce a detailed image of it and determine the positions and absolute brightness of more than 100 million celestial objects. When completed, it will provide a three-dimensional map of nearly a million galaxies and quasars. The total survey information is about 15 terabytes and rivals the information content of the Library of Congress. As the survey progresses, the data are released to the scientific community and the general public in annual increments.
Bandwidth Challenge Competitors: Winner
National Center for Data Mining (NCDM) at UIC, Northwestern Univ., Johns Hopkins Univ., "Transporting Sloan Digital Sky Survey Data Using SECTOR."
Honorable Mention
1) CalTech, CERN, Univ. of Florida, Univ. of Michigan, "High Speed Data Gathering, Distribution and Analysis for Physics Discoveries at the Large Hadron Collider"
2) Indiana Univ., Pittsburgh SuperComputing Center, Oak Ridge National Laboratory, "All in a Day's Work: Advancing Data Intensive Research with the Data Capacitor"
Additional teams
Japanese Aerospace Exploration Agency; Pacific Northwest National Laboratory; Purdue University; Internet2, Univ. of Washington; Texas A&M Univ., Univ. of Delaware; Univ. of Tokyo
(Digg, Technorati, more)
Like many organizations that rely on industrial-strength datacenters, the US Department of Energy (DOE) would like to know if cloud computing can make its life easier. To answer that question, the DOE is launching a $32 million program to study how scientific codes can make use of cloud technology.
Read More...
UT's Kraken breaks petascale barrier for academia; and Cyprus Institute partners with University of Illinois. We recap those stories and more in our weekly wrapup.
Read More...
While most of the industry has gone through an enormous consolidation over the past year, Amsterdam-based ClusterVision is defying the trend by spinning off its ClusterVisionOS HPC management software into a separate company called Bright Computing. The new business launched itself this week, along with its flagship offering, Bright Cluster Manager.
Read More...
Oct 14 | Linux Magazine | Nvidia's Fermi architecture is a game changer for GP-GPU. Read more...
Oct 14 | Computerworld | Apple alum Steve Wozniak finds a lot to like with Fusion-io. Read more...
Oct 14 | International Science Grid This Week | DOE supers used for nano-scale simulations. Read more...
Oct 12 | CNET News | Researchers grapple with Internet-scale data sets. Read more...
Oct 12 | InfoWorld | McNealy and Ellison vow to carry Sun technologies forward. Read more...
Sep 29 | | This white paper will outline the procedure for designing a green high performance HPC cluster using Appro GreenBlades™, Intel® Xeon® 5500 processor series (codename Nehalem), Arista 10 Gigabit Ethernet switches, and the Rocks+ software from Clustercorp.
Aug 05 | | Proven cloud or Infrastructure as a Service (IaaS) computing solutions such as IBM’s Computing on Demand (CoD) are very attractive for many commercial enterprises especially in today’s environment of flat or shrinking IT budgets. With a spectrum of flexible offerings and pricing models, IBM’s CoD solution provides secure, affordable, elastic, and risk-free access to IT infrastructure resources for companies that need to quickly scale-up or scale-down their IT needs to adapt to their business demands.
Escalating energy and operational costs of building and maintaining data centers are forcing enterprises to adopt cloud computing models. But are Infrastructure as a Service (IaaS) solutions like IBM's Computing on Demand (CoD) really cost effective? Join the discussion as industry experts discuss how you can exploit cloud computing for maximum ROI.
Sun Grid Engine software is the world's leading - and most widely deployed - distributed resource manager. The Sun Grid Engine software supports extreme scalability, simplifies administration and supports a diverse set of use case scenarios. It has industry-leading reliability and offers fast return on investment. Originally released by Sun in September 2000, the newest release is Sun Grid Engine 6.2 Update 3 and it is built on the Grid Engine open source project. Learn how to manage your workloads in an high performance computing environment and get your free white paper.
Source: Addison Snell, GM/VP, Tabor Research; sponsored by Dell
Many organizations that could benefit from the use of HPC clusters find that it is complicated to get the systems up and running because of limited IT resources or the complexities of the clusters themselves. Learn how the Intel Cluster Ready program, for which Dell was an original partner, seeks to address this challenge for entry level and mid-range HPC users.
