File size: 1,443 Bytes
c9573ac af69a4f c9573ac |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 |
Dataset stats: \
lat_mean = 39.951564548022596 \
lat_std = 0.0006361722351128644 \
lon_mean = -75.19150880602636 \
lon_std = 0.000611411894337979
The model can be loaded using:
```
from huggingface_hub import hf_hub_download
import torch
# Specify the repository and the filename of the model you want to load
repo_id = "FinalProj5190/ImageToGPSproject_new_vit" # Replace with your repo name
filename = "resnet_gps_regressor_complete.pth"
model_path = hf_hub_download(repo_id=repo_id, filename=filename)
# Load the model using torch
model_test = torch.load(model_path)
model_test.eval() # Set the model to evaluation mode
```
The model implementation is here:
```
class MultiModalModel(nn.Module):
def __init__(self, num_classes=2):
super(MultiModalModel, self).__init__()
self.vit = ViTModel.from_pretrained('google/vit-base-patch16-224-in21k')
# Replace for regression instead of classification
self.regression_head = nn.Sequential(
nn.Linear(self.vit.config.hidden_size, 512),
nn.ReLU(),
nn.Linear(512, num_classes)
)
def forward(self, x):
outputs = self.vit(pixel_values=x)
# Take the last hidden state (CLS token embedding)
cls_output = outputs.last_hidden_state[:, 0, :]
# Pass through the regression head
gps_coordinates = self.regression_head(cls_output)
return gps_coordinates
``` |