コンテンツにスキップ

General

Info

The command line examples on this page use the following notation
[login]$ : login node
[rNnN]$ : compute node
[login/rNnN]$ : login node or compute node
[yourPC]$ : environment from which the login node is connected

What is TSUBAME4.0?

TSUBAME 4.0 is a supercomputer operated and managed by Center for Information Infrastructure (CII) of the Institute of Science Tokyo. TSUBAME 4.0 has a theoretical calculation performance of 952 PFlops (half precision) and is expected to be the largest supercomputer in Japan handling a wide range of workloads including big data and AI in addition to conventional High Performance Computing.

For what purpose can TSUBAME be used?

Use of TSUBAME is limited to education, research, clerical work and social contribution purpose only. It can not be used for applications that directly lead to private financial interests. For example, mining virtual currency using block chain technology.

Conditions for using TSUBAME4.0

An account is required to use TSUBAME4.0.
Account types vary depending on affiliation and system. See How to apply for a TSUBAME account.

How to get started with TSUBAME4.0

This part shows the flow until setup an environment for running program.

There are 6 steps necessary to use TSUBAME4.

When step 1 and 2 are done, login is enabled. To submit jobs, you need to complete additional steps 3 - 5. In addition to your home directory of 25GiB, if you need additional volumes, do step 6.

  1. Getting an account
  2. SSH key pair generation and the public key registration
  3. Creation of a group (group administrator only)
  4. Addition of users to the group(group administrator and its members)
  5. Tsubame point purchase(group administrator only)
  6. Setup of group disk(group administrator only)

How to write acknowledgments in a paper using TSUBAME?

Please refer to the following page for an example of how to write an acknowledgement. Please note that this is just an example, and you may adjust the description to match the description of other supercomputers or research funds.

Please mention TSUBAME usage in acknowledgement of publications

In addition, please submit reports on your use of TSUBAME, such as bibliographic information, through TSUBAME Portal to help us understand how TSUBAME is being used. Please refer to the following User's Guide for how to submit usage reports.

TSUBAME portal User's Guide Management of TSUBAME usage report

Differences between login node and compute node

The difference between login node and compute node is as follows.

Login node Compute node
# of nodes 2 240
CPU AMD EPYC 7443 24-Core/2.85GHz x 2 AMD EPYC 9654 96-Core/2.4GHz x 2
Memory 256GiB 768GiB
GPU None NVIDIA H100 SXM5 × 4
Local storage None 1.92TB

The login nodes are shared servers and are not assumed to be used for calculation purpose. Please avoid high-load processing such as program execution at the login nodes, execute it on compute nodes though job scheduler.
Please refer to TSUBAME4.0 User's Guide for details.

I'm a beginner, I don't know what to do.

The content depends on what you are a beginner for.

Beginners of UNIX/LINUX

Upon using TSUBAME 4.0, users are required to master UNIX/Linux proficiently levels. Handbooks are made on this assumption.
If you do not understand the content of the handbooks, please read the UNIX/Limux beginner's book at the library, and understand how to use UNIX shells and commands.

Info

TSUBAME4.0 assumes that the participants are proficient enough to understand and practice the contents of “Introduction to TSUBAME (Linux basics)” on page 28 "File Operation" to page 62 "Module command to set environment" in the "Introduction to TSUBAME (Linux basics)" posted on Seminar text.

Beginners of supercomputer

This is for people who have used UNIX/Linux but have no experience of using supercomputers.

  • I don't know how to login to TSUBAME4.0
    Basically, SSH public key authentication method is used for remote login to TSUBAME4.0.
    For more information on "Remote login","SSH public key authentication method"and more, please see "Remote login" on page 21 to "Tips" on page 27 of "Introduction to TSUBAME (Linux basics)" posted on Seminar Text.
    It also introduces the SSH connection procedure using terminal software "PuTTy". Please refer to How to create an SSH key pair using PuTTY or MobaXterm and I want to know how to login to TSUBAME4.0 using PuTTY.
    TSUBAME4.0 also provides an Open OnDemand function that allows you to use the supercomputer easily via a web browser. For details, please refer to Open OnDemand usage guide.

  • I don't know how to perform compute processing on TSUBAME4.0
    TSUBAME4.0 uses a system called a job scheduler to perform processing on compute nodes.
    For more information on the concept of the job scheduler, please refer to pages 39 to 50 of the "TSUBAME4.0 Guidance" posted on Lectures. (Japanese only)
    For detailed usage, see Job Scheduler.
    In addition, please also refer to the "TSUBAME4.0 Guidance" posted on Lectures. (Japanese only)
    Once you have gained knowledge related to supercomputers, please also check the following it.

Beginners of TSUBAME 4.0

It is assumed that the user has experience of using TSUBAME 3.0 or other supercomputer sites.

  • Those with experience of using TSUBAME 3.0
    If you have been using TSUBAME 3, please refer to "Migration from TSUBAME 3".
    Please also refer to "TSUBAME4.0 Guidance" posted on Lectures.

  • Those with experience of using other supercomputer sites
    TSUBAME4.0 original specifications, rules and restrictions are summarised Quick Start Guide. Please read this before use.

Beginners of compile

If you have used UNIX/Linux but never used a commandline compile, please read Intel compiler or NVIDIA HPC SDK or AOCC.
In addition, if you use intel compiler, refer to "Parallel Programming" in Lectures.

Beginners of parallel programming(OpenMP,MPI)

Please refer to "Parallel Programming" posted on Lectures.

Beginners of GPU programming

TSUBAME 4.0 regularly conducts workshops, check the page of Lectures.
Please also refer to "GPU Programming" and "GPU Programming hands-on" posted on Lectures.

Beginners of ISV application software

Please check the application software guide for each. In addition, TSUBAME 3.0 regularly conducts workshops, check the page of Lectures.

About common errors in Linux

Here we have a FAQ on Linux common errors.
For details on how to use the described command, please check with the man command etc.

No such file or directory

There is no required file or directory.
It occurs when specifying a nonexistent file, directory name, etc., typing, or incorrect path specification.
Also, depending on the application, it may occur when the line feed code is CR + LF on windows.

Measures

Please review the file and directory name carefully. Also, please check FAQ "The job status is "Eqw" and it is not executed." about the newline character.

There are related errors as follows. 

error while loading shared libraries: ****.so: cannot open shared object file:  No such file or directory
This is an error that occurs when there is no library required by the program or can not be read.

Measures

Please check with ldd command. There is a way to set the environment variable LD_LIBRARY_PATH, explicitly specify the library at compile time, and so on.

command not found

The command you entered does not exist.
Depending on the type of program you wish to run, perform the following checks.

Example of adding the "hoge" directory directly under the home directory ($HOME) to the existing environment variable "PATH" 

[login/rNnN]$ export PATH=~/hoge:$PATH

Permission denied

You are not authorized to perform the operation you attempted to perform.
Linux and user and group permissions are set on a file / directory basis.
Check the authority of the target file or directory you want to read or write or execute with the following command.
(When checking the hoge file for an example) 

[login/rNnN]$ ls -l hoge

Measures
If you are trying to create files in / etc, / lib etc which are system directories etc, please make it in the user directory.
If it occurs in a user directory such as a group disk, check the authority and please do.

Disk quota exceeded

Please check FAQ How to solve "Disk quota exceeded" error"".

Out Of Memory

This error occurs when memory runs out.

Measures

  • Change the resource type to one with more memory capacity.
  • Divide the memory usage per node with mpi etc.

Related FAQ "Check the detail of an error message printed the log file"

About the IP address of the gateway server for compute nodes(connection to license servers outside TSUBAME, etc.)

The IP address range of the compute node gateway server is as follows. 

131.112.133.241, 131.112.133.242

When computing on TSUBAME by using a campus or university license server, please set so that communication within the above range is permitted.

Please keep in mind that the above address may be changed without notice from the circumstances of operation.

If your software requires communication with a license server outside of TSUBAME (e.g., in a laboratory), please confirm that you can communicate with the license server from a network outside of TSUBAME and outside of the license server before contacting us with the following information.

  • Global IP address of the license server
  • Port number of the license server (or all ports if there are more than one)
  • IP address of the host where the communication test was performed

I want to use pictures of TSUBAME in publication etc.

Some materials are available in a Box (Science Tokyo user only) for Science Tokyo faculties and staff. You do not need to obtain individual permission to use the materials under the following terms and conditions.

  • There is no need to add specific annotations such as copyright notices for use on the website or in printed materials.
  • The usage is limited to contexts related to the research using TSUBAME.
  • If you let your collaborators use the materials, Science Tokyo faculty or staff must download the materials and ensure that your collaborators comply with the terms of use.
  • Redistribution by anyone other than Science Tokyo faculty and staff is prohibited.

Please contact us individually if you are not affiliated with Science Tokyo, such as joint usage program users, or if you want materials not listed in the Box above.

If you use any of the above materials, we would appreciate it if you could report your use of them through the Contact Us page.

Other materials including the Science Tokyo logo are available at Science Tokyo Stock. (Science Tokyo user only). Please follow the rules and regulations of the Public Relations Office when using these materials.

I want to execute processing periodically on TSUBAME using cron or similar

The use of the cron function is prohibited in TSUBAME 4.0. Please note that setting crontab etc. will result in an error.
The TSUBAME4 login nodes are shared by many users and do not allow auto-restart program execution.

Can I use TSUBAME4 at my own expense (for personal use)?

There is no way to pay for TSUBAME4 usage at your own expense. Additionally, the TSUBAME Partnership Resource Allocations, which is a paid service for external users, only accepts applications from organizations such as universities, research institutions, and companies; we do not accept applications from individuals.
If you meet the requirements, please consider using the following programs

If you are affiliated with Science Tokyo

For Graduate Students

  • Graduate students at Science Tokyo who have an internal budget (e.g., from JSPS grants) can use that budget to pay for TSUBAME4 usage fees.
  • There is a program called TSUBAME Support Program for Young and Female Users that allows users under the age of 40 (with no age restriction for women), including graduate students, to use the service free of charge. Please consider applying.

For High School Students, Technical College Students, and Undergraduate Students

  • There is a program called TSUBAME's Support Program for Younger Users that allows high school students, technical college students, and undergraduate students to use the service free of charge. Please consider applying.

Procedures When the Science Tokyo Science & Engineering Field IC Card Expires Due to Extended Enrollment or Advancement

The necessary procedures and points to note when the Science Tokyo Science & Engineering Field IC Card expires are as follows.

If the IC card reissuance procedure was completed before the expiration date
You can continue using TSUBAME4.

If the IC card has expired and reissuance procedures are not completed
You cannot use TSUBAME4 until both the IC card reissuance procedure and the Account Procedures for Re-Issuance of Science Tokyo Science & Engineering Field IC Card after Expiration Date are completed.
Your home directory and work directory will not be immediately deleted, but data may be erased if reissuance is not completed for an extended period.
Group disks are managed by the respective group and are not affected by account suspension.

If the IC card reissuance procedure is completed after the expiration date
TSUBAME4 access will resume by performing the Account Procedures for Re-Issuance of Science Tokyo Science & Engineering Field IC Card after Expiration Date.
Upon resuming access, the previous account will be used as is.

Account Procedures for Re-Issuance of Science Tokyo Science & Engineering Field IC Card after Expiration Date

If your Science Tokyo Science & Engineering Field IC Card has expired and your card has been reissued, you can activate your account by logging in to Portal for Science & Engineering Field, and then logging in to the TSUBAME Portal page from “General Information Services” - “TSUBAME4 Portal”. General Information Services” - ‘TSUBAME4 Portal’ and log in to the TSUBAME Portal page to activate your account.
After your account is activated, you can login to TSUBAME Portal Page directly.