Upload folder using huggingface_hub

README.md

@@ -0,0 +1,104 @@
+---
+base_model:
+- buildborderless/CommunityForensics-DeepfakeDet-ViT
+library_name: transformers.js
+license: mit
+pipeline_tag: image-classification
+tags:
+- image-classification
+- timm
+- transformers
+- detection
+- deepfake
+- forensics
+- deepfake_detection
+- community
+- opensight
+---
+
+
+
+# CommunityForensics-DeepfakeDet-ViT (ONNX)
+
+
+This is an ONNX version of [buildborderless/CommunityForensics-DeepfakeDet-ViT](https://huggingface.co/buildborderless/CommunityForensics-DeepfakeDet-ViT). It was automatically converted and uploaded using [this Hugging Face Space](https://huggingface.co/spaces/onnx-community/convert-to-onnx).
+
+
+## Usage with Transformers.js
+
+
+See the pipeline documentation for `image-classification`: https://huggingface.co/docs/transformers.js/api/pipelines#module_pipelines.ImageClassificationPipeline
+
+
+---
+
+
+# Trained on 2.7M samples across 4,803 generators (see Training Data)
+
+Model presented in [Community Forensics: Using Thousands of Generators to Train Fake Image Detectors](https://huggingface.co/papers/2411.04125).
+
+**Uploaded for community validation as part of OpenSight** - An upcoming open-source framework for adaptive deepfake detection.
+
+**Project OpenSight HF Spaces coming soon with an eval playground and eventually a leaderboard. Preview:** 
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/639daf827270667011153fbc/AUmW697OefKN83BClM1ae.png)
+
+## Model Details
+### Model Description
+Vision Transformer (ViT) model trained on the largest dataset to-date for detecting AI-generated images in forensic applications.
+
+- **Developed by:** Jeongsoo Park and Andrew Owens, University of Michigan
+- **Model type:** Vision Transformer (ViT-Small)
+- **License:** MIT (compatible with CreativeML OpenRAIL-M referenced in [2411.04125v1.pdf])
+- **Finetuned from:** timm/vit_small_patch16_384.augreg_in21k_ft_in1k
+- **Adapted for HF** inference compatibility by AI Without Borders.
+
+**HF Space will be open sourced shortly showcasing various ways to run ultra-fast inference. Make sure to follow us for updates, as we will be releasing a slew of projects in the coming weeks.**
+
+### Links
+- **Repository:** [JeongsooP/Community-Forensics](https://github.com/JeongsooP/Community-Forensics)
+- **Paper:** [arXiv:2411.04125](https://arxiv.org/pdf/2411.04125)
+- **Project Page:** https://jespark.net/projects/2024/community_forensics
+
+## Training Details
+### Training Data
+- 2.7mil images from 15+ generators, 4600+ models
+- Over 1.15TB worth of images
+
+### Training Hyperparameters
+- **Framework:** PyTorch 2.0
+- **Precision:** bf16 mixed
+- **Optimizer:** AdamW (lr=5e-5)
+- **Epochs:** 10
+- **Batch Size:** 32
+
+## Evaluation
+### Unverified Testing Results
+- Only unverified because we currently lack resources to evaluate a dataset over 1.4T large.
+
+| Metric        | Value |
+|---------------|-------|
+| Accuracy      | 97.2% |
+| F1 Score      | 0.968 |
+| AUC-ROC       | 0.992 |
+| FP Rate       | 2.1%  |
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/639daf827270667011153fbc/g-dLzxLBw1RAuiplvFCxh.png)
+
+## Re-sampled and refined dataset
+
+- **Coming soon™**
+
+## Citation
+**BibTeX:**
+```bibtex
+@misc{park2024communityforensics,
+    title={Community Forensics: Using Thousands of Generators to Train Fake Image Detectors}, 
+    author={Jeongsoo Park and Andrew Owens},
+    year={2024},
+    eprint={2411.04125},
+    archivePrefix={arXiv},
+    primaryClass={cs.CV},
+    url={https://arxiv.org/abs/2411.04125}, 
+}
+```

config.json

@@ -0,0 +1,28 @@
+{
+  "architectures": [
+    "ViTForImageClassification"
+  ],
+  "attention_probs_dropout_prob": 0.0,
+  "dtype": "float32",
+  "encoder_stride": 16,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.0,
+  "hidden_size": 384,
+  "image_size": 384,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-06,
+  "mlp_ratio": 4,
+  "model_type": "vit",
+  "num_attention_heads": 12,
+  "num_channels": 3,
+  "num_classes": 1,
+  "num_heads": 6,
+  "num_hidden_layers": 12,
+  "num_layers": 12,
+  "patch_size": 16,
+  "pooler_act": "tanh",
+  "pooler_output_size": 384,
+  "qkv_bias": true,
+  "transformers_version": "4.57.6"
+}

onnx/model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:55602c73f457858f0b48c19bfa03bc6bd218f37027cb1f6b569ee24e2f954d03
+size 143845409

onnx/model_bnb4.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:263c46052167a15b981848465b8adb9f28dbd1f9ad8ecf8157cb05d876f7091b
+size 24416892

onnx/model_fp16.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2458c8472f3f93ecbda4acbe137382b559845f76ecf142f0fcbc03a07c7de739
+size 72106631

onnx/model_int8.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a504b8ea9372e8c9be9e8e3aa0a7f0f2eff5ba3df067c3d5ed2fcdfe15eaba2c
+size 36969938

onnx/model_q4.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:263c46052167a15b981848465b8adb9f28dbd1f9ad8ecf8157cb05d876f7091b
+size 24416892

onnx/model_q4f16.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ab58b5e202b4ad737dc7be5aae05f57deddb24c5c722d2d78e38adc913c33eb
+size 21234115

onnx/model_quantized.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a504b8ea9372e8c9be9e8e3aa0a7f0f2eff5ba3df067c3d5ed2fcdfe15eaba2c
+size 36969938

onnx/model_uint8.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:86097e4bec2e4ff1d8d5ae575c3ffa0ae55a080cbba2181879fa1a068659e2b2
+size 36969975

preprocessor_config.json

@@ -0,0 +1,25 @@
+{
+  "crop_pct": 0.875,
+  "crop_size": 384,
+  "do_convert_rgb": null,
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.48145466,
+    0.4578275,
+    0.40821073
+  ],
+  "image_processor_type": "ViTImageProcessor",
+  "image_std": [
+    0.26862954,
+    0.26130258,
+    0.27577711
+  ],
+  "resample": 3,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "height": 440,
+    "width": 440
+  }
+}