[에러] RuntimeError: CUDA out of memory.

feature extraction 하려는데 오류가 떳어요....ㅠㅠ

 

뚜둥!!!

 

CUDA out of memory !!!

 

RuntimeError: CUDA out of memory. Tried to allocate 12.50 MiB (GPU 0; 10.92 GiB total capacity; 8.57 MiB already allocated; 9.28 GiB free; 4.68 MiB cached)

 

열심히 구글링 해서 찾은 해결 방법!

 

1. batch size를 줄여라!

-> 안됨 ㅠㅠ

 

2. nvidia-smi 로 실행중인 프로세스를 보고, 죽여라!

kill -9 [PID]

ex) kill 6371

안됨......ㅠㅠ

 

3. Holding python variable 문제!

기존

features = []
for x, y in train_loader:
    x = torch.tensor(x, device=device, dtype=torch.float32)
    z = model(x)
    features.append(z)

수정

features = []
for x, y in train_loader:
    x = torch.tensor(x, device=device, dtype=torch.float32)
    z = model(x)
    features.append(z.tolist())

 

돌아간다!!!

 

Reference

github.com/pytorch/pytorch/issues/16417

RuntimeError: CUDA out of memory. Tried to allocate 12.50 MiB (GPU 0; 10.92 GiB total capacity; 8.57 MiB already allocated; 9.28