|
--- |
|
license: openrail |
|
--- |
|
|
|
# Monkey V4 Data Driven + Attention Readout Model Card |
|
|
|
Neural Encoding model for Macaque V4. The model is a combination of a data driven core and an attention readout layer. |
|
|
|
<p align="center"><img src="./assets/schematic.png" width="100%" alt="Data Driven V4 Schematic" /></p> |
|
|
|
## Model Details |
|
### Model Description |
|
|
|
This model is a combination of a data driven core and an attention readout layer. |
|
The data driven core is a convolutional neural network and the attention readout layer is a multihead attention layer with each head trained to predict the firing rates of a neuron in Macaque V4. |
|
|
|
### Model Sources |
|
For research purposes, we recommend our `nnvision` Github repository (https://github.com/sinzlab/nnvision), which contains the code for the model defintions and training. |
|
|
|
- Repository: https://github.com/sinzlab/nnvision |
|
- Paper: https://www.biorxiv.org/content/10.1101/2023.05.18.541176v1 |
|
|
|
### Intended Use |
|
|
|
The model is intended for research purposes only. |
|
|
|
### Model Use |
|
|
|
The model can be used to predict the firing rates of neurons in Macaque V4 given an image. |
|
|
|
#### nnvision |
|
The model can be used in Python with the `nnvision` package. |
|
```python |
|
import torch |
|
from nnvision.models.trained_models.v4_data_driven import v4_multihead_attention_ensemble_model |
|
|
|
input_image = torch.rand(1, 100, 100) |
|
firing_rate = v4_multihead_attention_ensemble_model(input_image, data_key="all_sessions") |
|
``` |
|
|
|
### energy-guided diffusion |
|
The model can be used in Python with the `energy-guided-diffusion` package. |
|
```python |
|
from egg.models import models |
|
model = models['data_driven']['train'] |
|
``` |
