Evaluating Success: Quadratic Weighted Kappa

The competition utilized a metric known as Quadratic Weighted Kappa (QWK) for evaluation. QWK assesses the level of agreement between two predictions, where 0 indicates random agreement and 1 signifies perfect agreement. A negative QWK suggests less agreement than would occur by chance. The calculation involves constructing an N x N histogram matrix to compare actual versus predicted values.

Source: Kaggle

Data Preprocessing: The Initial Challenge

Processing WSIs is notoriously labor-intensive. For my work, I employed OpenSlide to partition the slides into manageable tiles. The winning solution implemented a method called Concatenate Tile Pooling (CTP). While resizing images may seem appealing, it often results in significant information loss.

CTP allows for the selection of N tiles from each image based on tissue pixel content, processing these tiles through convolutional layers individually. The results are then combined into a comprehensive map prior to pooling and connecting to a fully connected head.

Source: Kaggle

Source: Github (CC license)

Addressing Dataset Noises with Image Hashing

The dataset included noise and duplicates, complicating the task. Imagehash, a library for generating hash values based on an image's visual content, was utilized to identify and remove duplicate images. The following code snippet demonstrates the process:

import imagehash

from tqdm import tqdm_notebook as tqdm

import cv2

import numpy as np

# Different hashing types

hashes = []

for path in tqdm(paths, total=len(paths)):

image = cv2.imread(path)

image = Image.fromarray(image)

hashes.append(np.array([f(image).hash for f in funcs]).reshape(256))

# Calculate similarity scores

sims = np.array([(hashes[i] == hashes).sum(dim=1).cpu().numpy()/256 for i in range(hashes.shape[0])])

threshold = 0.96

duplicates = np.where(sims > threshold)

Source: Kaggle

Modeling Techniques: EfficientNet and Beyond

The winning solution employed three different EfficientNet models and utilized Cross-Entropy Loss. EfficientNet has gained popularity for supervised image classification tasks, including its successful application in the Melanoma competition.

While many competitors relied on ensembles of two networks, this approach included an additional network for label cleaning, addressing the noisy dataset—one of the major challenges in the competition. This mirrors the pseudo-labeling technique used in other competitions.

Dynamic Learning Rate Management

Employing a cosine annealing scheduler, the learning rate fluctuates throughout the training phase, starting high and approaching zero before rising again, creating a cosine wave effect.

Source: Wikipedia commons

Enhancing Model Performance with Data Augmentation

Data augmentation has become a standard practice in leading Kaggle solutions. The team incorporated techniques such as cutout and mixup to enhance generalization. Cutout involves removing sections of input images, generating partially obscured versions to enrich the dataset.

Source: arXiv

Data Processing Insights

The effectiveness of data processing techniques cannot be overstated. Analyzing other solutions reveals that data cleaning was pivotal for this competition's winning entry.

To summarize the solution approach:

1. Segment images based on similarity and eliminate duplicates.
2. Train with noisy labels.
3. Mitigate noise using prediction and original label discrepancies.
4. Retrain the model without noise.
5. Combine models for final predictions.