zpn commited on
Commit
079b4c1
·
1 Parent(s): b274630

feat: transformers js example

Browse files
Files changed (1) hide show
  1. README.md +34 -0
README.md CHANGED
@@ -3081,6 +3081,40 @@ Note the small differences compared to the full 768-dimensional similarities.
3081
 
3082
  </details>
3083
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3084
  ## Training
3085
 
3086
  Click the Nomic Atlas map below to visualize a 5M sample of our contrastive pretraining data!
 
3081
 
3082
  </details>
3083
 
3084
+ ### Transformers.js
3085
+
3086
+ ```javascript
3087
+ import { pipeline } from '@xenova/transformers';
3088
+
3089
+ // Create a feature extraction pipeline
3090
+ const extractor = await pipeline('feature-extraction', 'nomic-ai/modernbert-embed-base', {
3091
+ quantized: false, // Comment out this line to use the quantized version
3092
+ });
3093
+
3094
+ // Compute sentence embeddings
3095
+ const texts = ['search_query: What is TSNE?', 'search_query: Who is Laurens van der Maaten?'];
3096
+ const embeddings = await extractor(texts, { pooling: 'mean', normalize: true });
3097
+ console.log(embeddings);
3098
+ ```
3099
+
3100
+ <details><summary>Click to see Transformers.js usage with different quantizations</summary>
3101
+
3102
+ ```javascript
3103
+ import { pipeline } from '@xenova/transformers';
3104
+
3105
+ // Create a feature extraction pipeline
3106
+ const extractor = await pipeline('feature-extraction', 'nomic-ai/modernbert-embed-base', {
3107
+ dtype: 'q4f16',
3108
+ });
3109
+
3110
+ // Compute sentence embeddings
3111
+ const texts = ['search_query: What is TSNE?', 'search_query: Who is Laurens van der Maaten?'];
3112
+ const embeddings = await extractor(texts, { pooling: 'mean', normalize: true });
3113
+ console.log(embeddings);
3114
+ ```
3115
+
3116
+
3117
+
3118
  ## Training
3119
 
3120
  Click the Nomic Atlas map below to visualize a 5M sample of our contrastive pretraining data!