TAIDE CAN HELP
Automatic Summarization
Write a Letter
Write an Article
Chinese-to-English Translation
English-to-Chinese Translation
Automatic Summarization
Write a Letter
Write an Article
Chinese-to-English Translation
English-to-Chinese Translation
TAIDE CAN HELP
Automatic Summarization
Write a Letter
Write an Article
Chinese-to-English Translation
English-to-Chinese Translation
PHASED GOALS
Model Size 7B
Computational Resources
Dedicated Computing Infrastructure for TAIDE
- 32 nodes allocated specifically for TAIDE project model training.
- Each node boasts 8 V100 32GB GPUs, ensuring substantial processing power.
Optimization with DeepSpeed 03
- Implementing the DeepSpeed 03 framework for efficient and optimized training of large Al models within the TAIDE project.
Scaled Training Capabilities for TAIDE Models
- For 7B Model Training: Deploying 8 nodes, totaling 64 V100 GPUs.
- For 13B Model Training: Expanding to 12-16 nodes, utilizing 96-128 V100 GPUs.
Trustworthy Pre-Training Data
- Importance of starting with reliable data for building a Base Model.
- Project initiated by the Taiwanese government, led by our team.
- Access to pre-training data facilitated by government resources.
- Collaboration with public media and government agencies yielded about 1.7 billion tokens.
- Achieved a substantial collection of 420,000 pieces of data.
Model Size 13B
Computational Resources
Dedicated Computing Infrastructure for TAIDE
- 32 nodes allocated specifically for TAIDE project model training.
- Each node boasts 8 V100 32GB GPUs, ensuring substantial processing power.
Optimization with DeepSpeed 03
- Implementing the DeepSpeed 03 framework for efficient and optimized training of large Al models within the TAIDE project.
Scaled Training Capabilities for TAIDE Models
- For 7B Model Training: Deploying 8 nodes, totaling 64 V100 GPUs.
- For 13B Model Training: Expanding to 12-16 nodes, utilizing 96-128 V100 GPUs.
Trustworthy Pre-Training Data
- Importance of starting with reliable data for building a Base Model.
- Project initiated by the Taiwanese government, led by our team.
- Access to pre-training data facilitated by government resources.
- Collaboration with public media and government agencies yielded about 1.7 billion tokens.
- Achieved a substantial collection of 420,000 pieces of data.
TAIDE Team
National Science and Technology Council
Comprising the National Applied Research Laboratories,
Ministry of Digital Affairs, Academia Sinica, professors from major
universities, and experts from various fields.
Elite Ensemble
Core model training with RLHF group
Superior Team
International observation with specification group
Evaluation system and Testing Environment Team
Project Supervision Management
Project Info
This project is grounded in Taiwanese culture, incorporating unique elements such as Taiwanese
language, values, and customs,
enabling generative AI to understand and respond to the needs of local users.
It aims to create a trustworthy foundational model for generative AI engines.
As AI applications become increasingly widespread, both businesses and the public have higher
expectations. Consequently, there is an urgent demand for AI's performance, safety and
robustness, as well as fairness and
transparency.
Custom-Built for Taiwan
Creating a trustworthy generative AI dialogue engine foundational model specifically for Taiwan, the government and industry can select the model size by computational power according to their services, to train the model themselves and establish internal applications.
Diverse Traditional Chinese Training Materials
Incorporating various thematic textual resources and training materials to create specific domain application examples, enhancing the model's performance in different thematic areas.
Laying the Foundation for Widespread Application
Constructing a computational environment and application service platform, providing the necessary computational power for model development, and laying the groundwork for subsequent promotion and application.
Strengthening the AI Development Environment
Through legal analysis, establishment of testing norms, and development of evaluation tools, the AI development environment is strengthened and public trust is enhanced.
Public-Private Collaboration for Mutual Benefit
Utilizing public-private collaboration to assist industries in integrating foundational models, these can be used for specific industrial applications. Custom adjustments are made with data provided by the businesses.
