pastebin.com

AI

language model training

SHOW HN2019

pastebin.com

I trained a GPT-2 1.5B Subreddit Simulator in 3 days using 88 TPUs

pastebin.com

#gpt-2#tpu training#language model#subreddit simulator

10points
3comments
#1,798of 2,195 in 2019
Top 82%of 2019 by points

How HN took it

QuietFewer than 5 comments about the project

0 positive, 1 neutral or asking, 0 mixed, 0 critical, out of 1 comments about the project.

The thread, comment by comment

Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.

positive neutral or mixed critical about another commenter or something else the maker

What HN said

What people liked

  • Swarm training across TPUs is cheaper than a TPU pod

    From 1 comment
    • This was done using preemptible TPUv3's, but you can get similar results with TPUv2's. It would just take about 3 times longer. I'm excited about the multi-TPU technique. We've…
      sillysaurusx on HN

Questions

  • How much did the parallel training cost?

    From 1 comment

Alternatives named

  • TPU pod

    From 1 comment
    • This was done using preemptible TPUv3's, but you can get similar results with TPUv2's. It would just take about 3 times longer. I'm excited about the multi-TPU technique. We've…
      sillysaurusx on HN

Category, keywords, summaries and labels on this page were made by AI (Thread Reader v2) and not checked by hand. Each point links to the comments it came from.

Add a badge to your READMEShown on HackerShow
[![Shown on HackerShow](https://hackershow.org/badge.svg)](https://hackershow.org/show/21877210)