Food Recognition Challenge
🍕 Food Recognition Challenge : Data Exploration & Baseline
Food Recognition Challenge Notebook
Shubhamai
🍕 Food Recognition Challenge : Data Exploration & Baseline
Problem¶
Detecting & Segmenting various kinds of food from an image. For ex. Someone got into new restaurent and get a food that he has never seen, well our DL model is in rescue, so our DL model will help indentifying which food it is from the class our model is being trained on!
Data¶
We will be using data from Food Recognition Challenge - A benchmark for image-based food recognition challange which was launched on March 9, 2020 and ended on May 26, 2020.
https://www.aicrowd.com/challenges/food-recognition-challenge#datasets
- We have a total of 24120 RGB images with 2053 validation, all in MS-COCO format and test set for now is same as validation ( debug mode ).
Evaluation¶
The evaluation metrics is IOU aka. Intersection Over Union ( more about that later ).
The actualy metric is computed by averaging over all the precision and recall values for IOU which greater than 0.5.
https://www.aicrowd.com/challenges/food-recognition-challenge#evaluation-criteria
Table of Content¶
- Setting our Workspace 💼
- Downloading & Unzipping our Dataset
- Downloading & Importing Necessary Libraries
Data Exploration 🧐
- Reading our Dataset
- Data Visualisations
Image Visulisation 🖼️
- Reading Images
Creating our Dataset 🔨
- Fixing the Dataset
- Creating our dataset
Creating our Model 🏭
- Creating R-CNN Model
- Setting up hyperparameters
Training the Model 🚂
- Setting up Tensorboard
- Start Training!
Evaluating the model 🧪
- Evaluating our Model
Testing the Model 💯
- Testing the Model
Submitting our predictions 📝
Generate More Data + Some tips & tricks 💡
In [ ]:
Setting our Workspace 💼¶
In this section we will be downloading our dataset, unzipping it & downliading detectron2 library and importing all libraries that we will be using
Downloading & Unzipping our Dataset¶
In [ ]:
# Downloading Training Dataset
!wget -q https://datasets.aicrowd.com/default/aicrowd-public-datasets/food-recognition-challenge/v0.4/train-v0.4.tar.gz -O train.zip
# Downloading Validation Dataset
!wget -q https://datasets.aicrowd.com/default/aicrowd-public-datasets/food-recognition-challenge/v0.4/val-v0.4.tar.gz -O val.zip
In [ ]:
# Unzipping Training Dataset
!unzip train.zip > /dev/null
In [ ]:
# Unzipping Validation Dataset
!unzip val.zip > /dev/null
So, the data structure is something like this
content
|
└─── sample_data
|
└─── Train
│ │ annotations.json
│ └───images
│ │ 012170.jpg
│ │ 012030.jpg
│ │ ...
│
└─── val
│ │ annotations.json
│ └───images
│ │ 011397.jpg
│ │ 012340.jpg
│ │ ...
| train.zip
| val.zipImporting Necessary Libraries¶
In [1]:
# Making sure that we are using GPUs
!nvidia-smi
In [ ]:
# install dependencies: (use cu101 because colab has CUDA 10.1)
!pip install -U torch==1.5 torchvision==0.6 -f https://download.pytorch.org/whl/cu101/torch_stable.html
!pip install cython pyyaml==5.1
!pip install -U 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'
import torch, torchvision
print(torch.__version__, torch.cuda.is_available())
!gcc --version
In [ ]:
# install detectron2:
!pip install detectron2==0.1.2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/index.html
In [ ]:
# You may need to restart your runtime prior to this, to let your installation take effect
# Some basic setup:
# Setup detectron2 logger
import detectron2
from detectron2.utils.logger import setup_logger
setup_logger()
# import some common libraries
import numpy as np
import pandas as pd
import cv2
import json
from tqdm.notebook import tqdm
# import some common detectron2 utilities
from detectron2 import model_zoo
from detectron2.engine import DefaultPredictor
from detectron2.config import get_cfg
from detectron2.utils.visualizer import Visualizer
from detectron2.data import MetadataCatalog
from detectron2.utils.visualizer import ColorMode
from detectron2.data.datasets import register_coco_instances
from detectron2.engine import DefaultTrainer
from detectron2.evaluation import COCOEvaluator, inference_on_dataset
from detectron2.data import build_detection_test_loader
# For reading annotations file
from pycocotools.coco import COCO
# utilities
from pprint import pprint # For beautiful print!
import os
# For data visualisation
import matplotlib.pyplot as plt
import plotly.graph_objects as go
import plotly.express as px
from google.colab.patches import cv2_imshow
In [ ]:
Data Exploration 🧐¶
In this section we are going to read our dataset & doing some data visualisations
Reading Data¶
In [ ]:
# Reading annotations.json
TRAIN_ANNOTATIONS_PATH = "/content/drive/MyDrive/aicrowd/annotations/training.json"
TRAIN_IMAGE_DIRECTIORY = "/content/train/images/"
VAL_IMAGE_DIRECTIORY = "/content/drive/MyDrive/aicrowd/annotations/validation.json"
VAL_ANNOTATIONS_PATH = "/content/val/annotations.json"
train_coco = COCO(TRAIN_ANNOTATIONS_PATH)
In [ ]:
# Reading the annotation files
with open(TRAIN_ANNOTATIONS_PATH) as f:
train_annotations_data = json.load(f)
with open(VAL_ANNOTATIONS_PATH) as f:
val_annotations_data = json.load(f)
In [ ]:
train_annotations_data['annotations'][0]
Data Format:
Our COCO data format is something like this -
"info": {...},
"categories": [...],
"images": [...],
"annotations": [...],In which categories is like this
[
{'id': 2578,
'name': 'water',
'name_readable': 'Water',
'supercategory': 'food'},
{'id': 1157,
'name': 'pear',
'name_readable': 'Pear',
'supercategory': 'food'},
...
{'id': 1190,
'name': 'peach',
'name_readable': 'Peach',
'supercategory': 'food'}
]Info is empty ( not sure why )
images is like this
[
{'file_name': '065537.jpg',
'height': 464,
'id': 65537,
'width': 464},
{'file_name': '065539.jpg',
'height': 464,
'id': 65539,
'width': 464},
...
{'file_name': '069900.jpg',
'height': 391,
'id': 69900,
'width': 392},
]Annotations is like this
{'area': 44320.0,
'bbox': [86.5, 127.49999999999999, 286.0, 170.0],
'category_id': 2578,
'id': 102434,
'image_id': 65537,
'iscrowd': 0,
'segmentation': [[235.99999999999997,
372.5,
169.0,
372.5,
...
368.5,
264.0,
371.5]]}In [ ]:
# Reading all classes
category_ids = train_coco.loadCats(train_coco.getCatIds())
category_names = [_["name_readable"] for _ in category_ids]
pprint(", ".join(category_names))
In [ ]:
category_names
Out[ ]:
In [ ]:
# Getting all categoriy with respective to their total images
no_images_per_category = {}
for n, i in enumerate(train_coco.getCatIds()):
imgIds = train_coco.getImgIds(catIds=i)
label = category_names[n]
no_images_per_category[label] = len(imgIds)
img_info = pd.DataFrame(train_coco.loadImgs(train_coco.getImgIds()))
no_images_per_category
Out[ ]:
In [ ]:
pd.DataFrame(no_images_per_category.items()).sort_values(1).iloc[::-1][0][:30].tolist()
Out[ ]:
Data Visualisations¶
In [ ]:
fig = go.Figure([go.Bar(x=list(no_images_per_category.keys()), y=list(no_images_per_category.values()))])
fig.update_layout(
title="No of Image per class",)
fig.show()
In [ ]:
pprint(f"Average number of image per class : { sum(list(no_images_per_category.values())) / len(list(no_images_per_category.values())) }")
pprint(f"Highest number of image per class is : { list(no_images_per_category.keys())[0]} of { list(no_images_per_category.values())[0] }")
pprint(f"Lowest number of image per class is : Veggie Burger of { sorted(list(no_images_per_category.values()))[0] }")
In [ ]:
fig = go.Figure(data=[go.Pie(labels=list(no_images_per_category.keys()), values=list(no_images_per_category.values()),
hole=.3, textposition='inside', )], )
fig.update_layout(
title="No of Image per class ( In pie )",)
fig.show()
In [ ]:
fig = go.Figure()
fig.add_trace(go.Histogram(x=img_info['height']))
fig.add_trace(go.Histogram(x=img_info['width']))
# Overlay both histograms
fig.update_layout(barmode='stack', title="Histogram of Image width & height",)
fig.show()
Image Visulisation 🖼️¶
In this section we are going to do imaghe visualisations!
In [ ]:
img_info
Out[ ]:
In [ ]:
len(train_annotations_data['annotations'][n]['segmentation']), len(train_annotations_data['annotations'][n]['bbox'])
Out[ ]:
In [ ]:
for n, i in enumerate(tqdm((train_annotations_data['annotations']))):
# if np.array(train_annotations_data['annotations'][n]['segmentation']).shape[0] != np.array(train_annotations_data['annotations'][n]['bbox']).shape[0]:
# print(n)
if np.array(train_annotations_data['annotations'][n]['segmentation']).shape[0] != 1:
print(n)
else:
pass
In [ ]:
img_no = 4
annIds = train_coco.getAnnIds(imgIds=train_annotations_data['annotations'][img_no]['image_id'])
anns = train_coco.loadAnns(annIds)
# load and render the image
plt.imshow(plt.imread(TRAIN_IMAGE_DIRECTIORY+train_annotations_data['images'][img_no]['file_name']))
plt.axis('off')
# Render annotations on top of the image
train_coco.showAnns(anns)
In [ ]:
w, h = 12, 12 # Setting width and height of every image
rows, cols = 5, 5 # Setting the number of image rows & cols
fig = plt.figure(figsize=(20, 14)) # Making the figure with size
plt.title("Images")
plt.axis('off')
# Going thought every cell in rows and cols
for i in range(1, cols * rows+1):
annIds = train_coco.getAnnIds(imgIds=img_info['id'][i])
anns = train_coco.loadAnns(annIds)
fig.add_subplot(rows, cols, i)
# Show the image
img = plt.imread(TRAIN_IMAGE_DIRECTIORY+img_info['file_name'][i])
for i in anns:
[x,y,w,h] = i['bbox']
cv2.rectangle(img, (int(x), int(y)), (int(x+h), int(y+w)), (255,0,0), 5)
plt.imshow(img)
# Render annotations on top of the image
train_coco.showAnns(anns)
# Setting the axis off
plt.axis("off")
# Showing the figure
plt.show()