Top 13 Deep Learning Tips and Tricks

  1. Understand model complexity
    • how does data affect model: complexity depends only on the first layer size
  2. Establish a baseline on holdout data: try different algos/ models and check
  3. Inspect model in flow
  4. Use early stopping for deep learning! Use a validation set
  5. Control scoring overhead: a smaller validation dataset and sample it, even score less
  6. Use N-fold CV: get directly the optimal training time with early stopping
  7. Perform hyper-parameter search: you just need to find one of the good models
  8. Use checkpointing: save your model in your disk when need!
  9. Tune communication on multi-node
  10. Know your data
  11. Advanced math in proba
  12. Do ensembling: vote or just weight!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s