Exploring Google Colab (Part 1 of N)

Written on March 14, 2019
[ deep-learning  machine-learning  python  tensorflow  easi  ]

One: this is called “Part 1 of N” b/c I know I’ll be posting more on this, and don’t want to have to come up w/ crazy, catchy titles for each, and I don’t know how many there will be! Two: on a side note, I originally dated this file as 2018 instead of 2019, which I’ve done for almost every 2019 post so far. It’s 2019, Kev! Get with it! Three: it’s Pi Day. I love math and physics so much, but Pi Day? Meh.

In this video, Getting Started with TensorFlow in Google Colaboratory, we start in Google Drive and create a new Colab JNB. The instructor pip installs tensorflow… I thought, “Wait, last time I checked, TensorFlow came pre-installed.” So I tested this out: went to Google Drive, created a new Colab JNB, and successfully imported tensorflow (import tensorflow as tf).

It then occurred to me: “But which version of TensorFlow is it?”

tf.__version__
  '1.13.1'

And it further occurred to me: “What is my default Python environment like in general?” It’s actually pretty jam packed.

!pip freeze
  absl-py==0.7.0
  alabaster==0.7.12
  albumentations==0.1.12
  altair==2.4.1
  astor==0.7.1
  astropy==3.0.5
  atari-py==0.1.7
  atomicwrites==1.3.0
  attrs==19.1.0
  audioread==2.1.6
  autograd==1.2
  Babel==2.6.0
  backcall==0.1.0
  backports.tempfile==1.0
  backports.weakref==1.0.post1
  beautifulsoup4==4.6.3
  bleach==3.1.0
  bokeh==1.0.4
  boto==2.49.0
  boto3==1.9.106
  botocore==1.12.106
  Bottleneck==1.2.1
  bs4==0.0.1
  bz2file==0.98
  cachetools==3.1.0
  certifi==2018.11.29
  cffi==1.12.2
  chainer==5.0.0
  chardet==3.0.4
  Click==7.0
  cloudpickle==0.6.1
  cmake==3.12.0
  colorlover==0.3.0
  community==1.0.0b1
  contextlib2==0.5.5
  convertdate==2.1.3
  coverage==3.7.1
  coveralls==0.5
  crcmod==1.7
  cufflinks==0.14.6
  cvxopt==1.2.3
  cvxpy==1.0.15
  cycler==0.10.0
  cymem==2.0.2
  Cython==0.29.6
  cytoolz==0.9.0.1
  daft==0.0.4
  dask==0.20.2
  dataclasses==0.6
  datascience==0.10.6
  decorator==4.3.2
  defusedxml==0.5.0
  dill==0.2.9
  distributed==1.25.3
  Django==2.1.7
  dlib==19.16.0
  dm-sonnet==1.23
  docopt==0.6.2
  docutils==0.14
  dopamine-rl==1.0.5
  easydict==1.9
  ecos==2.0.7.post1
  editdistance==0.5.2
  en-core-web-sm==2.0.0
  entrypoints==0.3
  enum34==1.1.6
  ephem==3.7.6.0
  et-xmlfile==1.0.1
  fa2==0.3.5
  fancyimpute==0.4.2
  fastai==1.0.46
  fastcache==1.0.2
  fastdtw==0.3.2
  fastprogress==0.1.20
  fastrlock==0.4
  fbprophet==0.4.post2
  featuretools==0.4.1
  filelock==3.0.10
  fix-yahoo-finance==0.0.22
  Flask==1.0.2
  folium==0.2.1
  future==0.16.0
  gast==0.2.2
  GDAL==2.2.2
  gdown==3.6.4
  gensim==3.6.0
  geographiclib==1.49
  geopy==1.17.0
  gevent==1.4.0
  gin-config==0.1.4
  glob2==0.6
  google==2.0.1
  google-api-core==1.8.0
  google-api-python-client==1.6.7
  google-auth==1.4.2
  google-auth-httplib2==0.0.3
  google-auth-oauthlib==0.2.0
  google-cloud-bigquery==1.8.1
  google-cloud-core==0.29.1
  google-cloud-language==1.0.2
  google-cloud-storage==1.13.2
  google-cloud-translate==1.3.3
  google-colab==1.0.0
  google-resumable-media==0.3.2
  googleapis-common-protos==1.5.8
  googledrivedownloader==0.3
  graph-nets==1.0.3
  graphviz==0.10.1
  greenlet==0.4.15
  grpcio==1.15.0
  gspread==3.0.1
  gspread-dataframe==3.0.2
  gunicorn==19.9.0
  gym==0.10.11
  h5py==2.8.0
  HeapDict==1.0.0
  holidays==0.9.9
  html5lib==1.0.1
  httpimport==0.5.16
  httplib2==0.11.3
  humanize==0.5.1
  hyperopt==0.1.2
  ideep4py==2.0.0.post3
  idna==2.6
  image==1.5.27
  imageio==2.4.1
  imagesize==1.1.0
  imbalanced-learn==0.4.3
  imblearn==0.0
  imgaug==0.2.8
  imutils==0.5.2
  inflect==2.1.0
  intel-openmp==2019.0
  intervaltree==2.1.0
  ipykernel==4.6.1
  ipython==5.5.0
  ipython-genutils==0.2.0
  ipython-sql==0.3.9
  ipywidgets==7.4.2
  itsdangerous==1.1.0
  jdcal==1.4
  jedi==0.13.3
  jieba==0.39
  Jinja2==2.10
  jmespath==0.9.4
  joblib==0.12.5
  jpeg4py==0.1.4
  jsonschema==2.6.0
  jupyter==1.0.0
  jupyter-client==5.2.4
  jupyter-console==6.0.0
  jupyter-core==4.4.0
  kaggle==1.5.3
  kapre==0.1.3.1
  Keras==2.2.4
  Keras-Applications==1.0.7
  Keras-Preprocessing==1.0.9
  keras-vis==0.4.1
  kiwisolver==1.0.1
  knnimpute==0.1.0
  librosa==0.6.3
  lightgbm==2.2.3
  llvmlite==0.27.1
  lmdb==0.94
  lucid==0.3.8
  lunardate==0.2.0
  lxml==4.2.6
  magenta==0.3.19
  Markdown==3.0.1
  MarkupSafe==1.1.1
  matplotlib==3.0.3
  matplotlib-venn==0.11.5
  mesh-tensorflow==0.0.5
  mido==1.2.6
  mir-eval==0.5
  missingno==0.4.1
  mistune==0.8.4
  mkl==2019.0
  mlxtend==0.14.0
  mock==2.0.0
  more-itertools==6.0.0
  moviepy==0.2.3.5
  mpi4py==3.0.1
  mpmath==1.1.0
  msgpack==0.5.6
  msgpack-numpy==0.4.3.2
  multiprocess==0.70.7
  multitasking==0.0.7
  murmurhash==1.0.2
  music21==5.5.0
  natsort==5.5.0
  nbconvert==5.4.1
  nbformat==4.4.0
  networkx==2.2
  nibabel==2.3.3
  nltk==3.2.5
  nose==1.3.7
  notebook==5.2.2
  np-utils==0.5.10.0
  numba==0.40.1
  numexpr==2.6.9
  numpy==1.14.6
  nvidia-ml-py3==7.352.0
  oauth2client==4.1.3
  oauthlib==3.0.1
  okgrade==0.4.3
  olefile==0.46
  opencv-contrib-python==3.4.3.18
  opencv-python==3.4.5.20
  openpyxl==2.5.9
  osqp==0.5.0
  packaging==19.0
  pandas==0.22.0
  pandas-datareader==0.7.0
  pandas-gbq==0.4.1
  pandas-profiling==1.4.1
  pandocfilters==1.4.2
  parso==0.3.4
  pathlib==1.0.1
  patsy==0.5.1
  pbr==5.1.3
  pexpect==4.6.0
  pickleshare==0.7.5
  Pillow==4.1.1
  pip-tools==3.4.0
  plac==0.9.6
  plotly==3.6.1
  pluggy==0.7.1
  portpicker==1.2.0
  prefetch-generator==1.0.1
  preshed==2.0.1
  pretty-midi==0.2.8
  prettytable==0.7.2
  progressbar2==3.38.0
  prometheus-client==0.6.0
  promise==2.2.1
  prompt-toolkit==1.0.15
  protobuf==3.6.1
  psutil==5.4.8
  psycopg2==2.7.6.1
  ptyprocess==0.6.0
  py==1.8.0
  pyasn1==0.4.5
  pyasn1-modules==0.2.4
  pycocotools==2.0.0
  pycparser==2.19
  pydot==1.3.0
  pydot-ng==2.0.0
  pydotplus==2.0.2
  pyemd==0.5.1
  pyglet==1.3.2
  Pygments==2.1.3
  pygobject==3.26.1
  pymc3==3.6
  pymongo==3.7.2
  pymystem3==0.2.0
  PyOpenGL==3.1.0
  pyparsing==2.3.1
  pyrsistent==0.14.11
  pysndfile==1.3.2
  PySocks==1.6.8
  pystache==0.5.4
  pystan==2.18.1.0
  pytest==3.6.4
  python-apt==1.6.3+ubuntu1
  python-chess==0.23.11
  python-dateutil==2.5.3
  python-louvain==0.13
  python-rtmidi==1.2.1
  python-slugify==3.0.0
  python-utils==2.3.0
  pytz==2018.9
  PyWavelets==1.0.2
  PyYAML==3.13
  pyzmq==17.0.0
  qtconsole==4.4.3
  regex==2018.1.10
  requests==2.18.4
  requests-oauthlib==1.2.0
  resampy==0.2.1
  retrying==1.3.3
  rpy2==2.9.5
  rsa==4.0
  s3fs==0.2.0
  s3transfer==0.2.0
  scikit-image==0.13.1
  scikit-learn==0.20.3
  scipy==1.1.0
  screen-resolution-extra==0.0.0
  scs==2.0.2
  seaborn==0.7.1
  Send2Trash==1.5.0
  setuptools-git==1.2
  Shapely==1.6.4.post2
  simplegeneric==0.8.1
  six==1.11.0
  sklearn==0.0
  smart-open==1.8.0
  snowballstemmer==1.2.1
  sortedcontainers==2.1.0
  spacy==2.0.18
  Sphinx==1.8.4
  sphinxcontrib-websupport==1.1.0
  SQLAlchemy==1.2.18
  sqlparse==0.2.4
  stable-baselines==2.2.1
  statsmodels==0.8.0
  sympy==1.1.1
  tables==3.4.4
  tabulate==0.8.3
  tblib==1.3.2
  tensor2tensor==1.11.0
  tensorboard==1.13.0
  tensorboardcolab==0.0.22
  tensorflow==1.13.1
  tensorflow-estimator==1.13.0
  tensorflow-hub==0.3.0
  tensorflow-metadata==0.12.1
  tensorflow-probability==0.6.0
  termcolor==1.1.0
  terminado==0.8.1
  testpath==0.4.2
  text-unidecode==1.2
  textblob==0.15.3
  textgenrnn==1.4.1
  tfds-nightly==1.0.1.dev201903040105
  tflearn==0.3.2
  Theano==1.0.4
  thinc==6.12.1
  toolz==0.9.0
  torch==1.0.1.post2
  torchsummary==1.5.1
  torchtext==0.3.1
  torchvision==0.2.2.post3
  tornado==4.5.3
  tqdm==4.28.1
  traitlets==4.3.2
  tweepy==3.6.0
  typing==3.6.6
  tzlocal==1.5.1
  ujson==1.35
  umap-learn==0.3.7
  uritemplate==3.0.0
  urllib3==1.22
  vega-datasets==0.7.0
  wcwidth==0.1.7
  webencodings==0.5.1
  Werkzeug==0.14.1
  widgetsnbextension==3.4.2
  wordcloud==1.5.0
  wrapt==1.10.11
  xarray==0.11.3
  xgboost==0.7.post4
  xkit==0.0.0
  xlrd==1.1.0
  xlwt==1.3.0
  yellowbrick==0.9.1
  zict==0.1.3
  zmq==0.0.0

Phew! Whoa! Jeez!

That’s every machine learning, statistics, visualization, and web/html oriented package I’ve ever used, read about, and heard of – plus more! This is no MiniConda :-p

It’s amazing, but you know once you go to update a package, things will start shifting: depedencies will be upgraded and downgraded. So, be wary.

Isn’t TensorFlow on version 2.0? When I try to update TensorFlow, it basically tells me everything is up to date:

!pip install tensorflow --upgrade

However, I was able to pip install the GPU version of TensorFlow, despite not using GPU hardware:

!pip install tensorflow-gpu

That might make you wonder:

  • what is my runtime environment?
  • what version of Python am I using
  • does my JNB have access to a GPU?

To answer these questions, click on the “Runtime” dropdown in your Colab notebook. You can reset your environment, choose what version of Python you want, and even choose whether you want to use GPUs or TPUs. All for free. Pretty crazy.

Some Refs

  • https://research.google.com/seedbank/
    • Really want to get to working through a bunch of these, then transforming them into new projects
  • https://www.youtube.com/playlist?list=PLQY2H8rRoyvwLbzbnKJ59NkZvQAW9wLbx
    • Also want to keep going through this playslist!