Hugging Face's logo

prithivida
/
Splade_PP_en_v1

Fill-Mask
Transformers
PyTorch
ONNX
English
bert
splade++
document-expansion
sparse representation
bag-of-words
passage-retrieval
knowledge-distillation
document encoder
Inference Endpoints
Model card Files
Splade_PP_en_v1
File size: 37,541 Bytes 
ab3bab5
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
 
[INFO|modeling_utils.py:2868] 2024-02-12 11:15:07,125 >> All the weights of BertSplade were initialized from the model checkpoint at ./data/splade/english-marco/train/distill/checkpoint-157165.
If your task is similar to the task the model of the checkpoint was trained on, you can already use BertSplade for predictions without further training.
[INFO|configuration_utils.py:505] 2024-02-12 11:15:07,137 >> loading configuration file ./data/splade/english-marco/train/distill/checkpoint-157165/generation_config.json


Sorting Corpus by document length (Longest first)...
Use len(str.split()) to sort the corpus for efficiently encoding
encoding: 100% 1/1 [00:07<00:00,  7.58s/it]
#> Joining saver thread.
8841823it [00:16, 544598.74it/s]
Sorting Corpus by document length (Longest first)...
Use len(str.split()) to sort the corpus for efficiently encoding
encoding: 100% 89/89 [6:03:47<00:00, 245.26s/it]
#> Joining saver thread.

2024-02-12 17:20:01,587 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:204) - Setting log level to INFO
2024-02-12 17:20:01,589 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:207) - ============ Loading Index Configuration ============
2024-02-12 17:20:01,589 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:208) - AbstractIndexer settings:
2024-02-12 17:20:01,590 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:209) -  + DocumentCollection path: ./data/splade/english-marco/train/distill/checkpoint-157165/evaluate/corpus/encoded
2024-02-12 17:20:01,590 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:210) -  + CollectionClass: JsonVectorCollection
2024-02-12 17:20:01,591 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:211) -  + Index path: ./data/splade/english-marco/train/distill/checkpoint-157165/evaluate/corpus
2024-02-12 17:20:01,591 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:212) -  + Threads: 32
2024-02-12 17:20:01,591 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:213) -  + Optimize (merge segments)? false
2024-02-12 17:20:01,737 INFO  [main] index.IndexCollection (IndexCollection.java:197) - IndexCollection settings:
2024-02-12 17:20:01,738 INFO  [main] index.IndexCollection (IndexCollection.java:198) -  + Generator: DefaultLuceneDocumentGenerator
2024-02-12 17:20:01,738 INFO  [main] index.IndexCollection (IndexCollection.java:199) -  + Language: en
2024-02-12 17:20:01,738 INFO  [main] index.IndexCollection (IndexCollection.java:200) -  + Stemmer: porter
2024-02-12 17:20:01,739 INFO  [main] index.IndexCollection (IndexCollection.java:201) -  + Keep stopwords? false
2024-02-12 17:20:01,739 INFO  [main] index.IndexCollection (IndexCollection.java:202) -  + Stopwords: null
2024-02-12 17:20:01,739 INFO  [main] index.IndexCollection (IndexCollection.java:203) -  + Store positions? false
2024-02-12 17:20:01,740 INFO  [main] index.IndexCollection (IndexCollection.java:204) -  + Store docvectors? false
2024-02-12 17:20:01,740 INFO  [main] index.IndexCollection (IndexCollection.java:205) -  + Store document "contents" field? false
2024-02-12 17:20:01,740 INFO  [main] index.IndexCollection (IndexCollection.java:206) -  + Store document "raw" field? false
2024-02-12 17:20:01,741 INFO  [main] index.IndexCollection (IndexCollection.java:207) -  + Additional fields to index: []
2024-02-12 17:20:01,741 INFO  [main] index.IndexCollection (IndexCollection.java:208) -  + Whitelist: null
2024-02-12 17:20:01,742 INFO  [main] index.IndexCollection (IndexCollection.java:209) -  + Pretokenized?: true
2024-02-12 17:20:01,742 INFO  [main] index.IndexCollection (IndexCollection.java:210) -  + Codec: Lucene99
2024-02-12 17:20:01,742 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:237) - ============ Indexing Collection ============
2024-02-12 17:20:01,744 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:246) - Thread pool with 32 threads initialized.
2024-02-12 17:20:01,744 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:247) - 1 file found in ./data/splade/english-marco/train/distill/checkpoint-157165/evaluate/corpus/encoded
2024-02-12 17:20:01,744 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:248) - Starting to index...
2024-02-12 17:21:01,753 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 40,000 documents indexed
2024-02-12 17:22:01,756 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 80,000 documents indexed
2024-02-12 17:23:01,759 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 130,000 documents indexed
2024-02-12 17:24:01,761 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 180,000 documents indexed
2024-02-12 17:25:01,763 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 220,000 documents indexed
2024-02-12 17:26:01,765 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 270,000 documents indexed
2024-02-12 17:27:01,767 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 320,000 documents indexed
2024-02-12 17:28:01,769 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 370,000 documents indexed
2024-02-12 17:29:01,771 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 420,000 documents indexed
2024-02-12 17:30:01,773 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 460,000 documents indexed
2024-02-12 17:31:01,776 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 510,000 documents indexed
2024-02-12 17:32:01,779 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 560,000 documents indexed
2024-02-12 17:33:01,781 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 610,000 documents indexed
2024-02-12 17:34:01,783 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 650,000 documents indexed
2024-02-12 17:35:01,785 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 700,000 documents indexed
2024-02-12 17:36:01,787 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 750,000 documents indexed
2024-02-12 17:37:01,790 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 800,000 documents indexed
2024-02-12 17:38:01,794 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 850,000 documents indexed
2024-02-12 17:39:01,797 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 900,000 documents indexed
2024-02-12 17:40:01,799 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 950,000 documents indexed
2024-02-12 17:41:01,801 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 1,000,000 documents indexed
2024-02-12 17:42:01,803 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 1,050,000 documents indexed
2024-02-12 17:43:01,806 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 1,100,000 documents indexed
2024-02-12 17:44:01,808 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 1,150,000 documents indexed
2024-02-12 17:45:01,810 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 1,200,000 documents indexed
2024-02-12 17:46:01,813 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 1,250,000 documents indexed
2024-02-12 17:47:01,815 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 1,300,000 documents indexed
2024-02-12 17:48:01,819 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 1,350,000 documents indexed
2024-02-12 17:49:01,822 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 1,400,000 documents indexed
2024-02-12 17:50:01,825 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 1,440,000 documents indexed
2024-02-12 17:51:01,827 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 1,490,000 documents indexed
2024-02-12 17:52:01,830 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 1,540,000 documents indexed
2024-02-12 17:53:01,832 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 1,590,000 documents indexed
2024-02-12 17:54:01,835 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 1,640,000 documents indexed
2024-02-12 17:55:01,839 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 1,690,000 documents indexed
2024-02-12 17:56:01,842 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 1,740,000 documents indexed
2024-02-12 17:57:01,845 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 1,790,000 documents indexed
2024-02-12 17:58:01,848 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 1,840,000 documents indexed
2024-02-12 17:59:01,851 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 1,890,000 documents indexed
2024-02-12 18:00:01,854 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 1,940,000 documents indexed
2024-02-12 18:01:01,856 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 1,990,000 documents indexed
2024-02-12 18:02:01,859 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 2,040,000 documents indexed
2024-02-12 18:03:01,861 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 2,090,000 documents indexed
2024-02-12 18:04:01,864 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 2,140,000 documents indexed
2024-02-12 18:05:01,866 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 2,200,000 documents indexed
2024-02-12 18:06:01,869 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 2,250,000 documents indexed
2024-02-12 18:07:01,871 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 2,300,000 documents indexed
2024-02-12 18:08:01,874 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 2,360,000 documents indexed
2024-02-12 18:09:01,878 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 2,410,000 documents indexed
2024-02-12 18:10:01,881 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 2,460,000 documents indexed
2024-02-12 18:11:01,883 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 2,520,000 documents indexed
2024-02-12 18:12:01,886 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 2,570,000 documents indexed
2024-02-12 18:13:01,888 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 2,630,000 documents indexed
2024-02-12 18:14:01,891 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 2,680,000 documents indexed
2024-02-12 18:15:01,894 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 2,740,000 documents indexed
2024-02-12 18:16:01,896 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 2,790,000 documents indexed
2024-02-12 18:17:01,899 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 2,850,000 documents indexed
2024-02-12 18:18:01,901 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 2,900,000 documents indexed
2024-02-12 18:19:01,905 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 2,960,000 documents indexed
2024-02-12 18:20:01,908 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 3,010,000 documents indexed
2024-02-12 18:21:01,911 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 3,070,000 documents indexed
2024-02-12 18:22:01,914 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 3,120,000 documents indexed
2024-02-12 18:23:01,917 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 3,180,000 documents indexed
2024-02-12 18:24:01,920 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 3,230,000 documents indexed
2024-02-12 18:25:01,924 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 3,290,000 documents indexed
2024-02-12 18:26:01,926 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 3,340,000 documents indexed
2024-02-12 18:27:01,929 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 3,400,000 documents indexed
2024-02-12 18:28:01,931 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 3,450,000 documents indexed
2024-02-12 18:29:01,935 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 3,510,000 documents indexed
2024-02-12 18:30:01,938 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 3,560,000 documents indexed
2024-02-12 18:31:01,940 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 3,610,000 documents indexed
2024-02-12 18:32:01,943 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 3,670,000 documents indexed
2024-02-12 18:33:01,945 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 3,720,000 documents indexed
2024-02-12 18:34:01,948 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 3,780,000 documents indexed
2024-02-12 18:35:01,951 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 3,830,000 documents indexed
2024-02-12 18:36:01,954 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 3,890,000 documents indexed
2024-02-12 18:37:01,958 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 3,940,000 documents indexed
2024-02-12 18:38:01,961 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 3,990,000 documents indexed
2024-02-12 18:39:01,964 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 4,050,000 documents indexed
2024-02-12 18:40:01,967 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 4,100,000 documents indexed
2024-02-12 18:41:01,969 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 4,160,000 documents indexed
2024-02-12 18:42:01,972 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 4,210,000 documents indexed
2024-02-12 18:43:01,975 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 4,260,000 documents indexed
2024-02-12 18:44:01,978 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 4,320,000 documents indexed
2024-02-12 18:45:01,981 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 4,370,000 documents indexed
2024-02-12 18:46:01,985 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 4,430,000 documents indexed
2024-02-12 18:47:01,988 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 4,480,000 documents indexed
2024-02-12 18:48:01,991 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 4,530,000 documents indexed
2024-02-12 18:49:01,995 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 4,590,000 documents indexed
2024-02-12 18:50:01,998 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 4,640,000 documents indexed
2024-02-12 18:51:02,001 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 4,700,000 documents indexed
2024-02-12 18:52:02,004 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 4,750,000 documents indexed
2024-02-12 18:53:02,009 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 4,810,000 documents indexed
2024-02-12 18:54:02,012 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 4,860,000 documents indexed
2024-02-12 18:55:02,015 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 4,910,000 documents indexed
2024-02-12 18:56:02,018 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 4,970,000 documents indexed
2024-02-12 18:57:02,022 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 5,020,000 documents indexed
2024-02-12 18:58:02,027 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 5,080,000 documents indexed
2024-02-12 18:59:02,030 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 5,130,000 documents indexed
2024-02-12 19:00:02,033 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 5,190,000 documents indexed
2024-02-12 19:01:02,038 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 5,240,000 documents indexed
2024-02-12 19:02:02,043 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 5,290,000 documents indexed
2024-02-12 19:03:02,048 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 5,350,000 documents indexed
2024-02-12 19:04:02,051 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 5,400,000 documents indexed
2024-02-12 19:05:02,055 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 5,460,000 documents indexed
2024-02-12 19:06:02,057 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 5,510,000 documents indexed
2024-02-12 19:07:02,061 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 5,570,000 documents indexed
2024-02-12 19:08:02,065 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 5,620,000 documents indexed
2024-02-12 19:09:02,068 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 5,680,000 documents indexed
2024-02-12 19:10:02,071 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 5,730,000 documents indexed
2024-02-12 19:11:02,074 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 5,790,000 documents indexed
2024-02-12 19:12:02,077 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 5,840,000 documents indexed
2024-02-12 19:13:02,081 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 5,900,000 documents indexed
2024-02-12 19:14:02,084 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 5,950,000 documents indexed
2024-02-12 19:15:02,086 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 6,010,000 documents indexed
2024-02-12 19:16:02,090 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 6,060,000 documents indexed
2024-02-12 19:17:02,093 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 6,120,000 documents indexed
2024-02-12 19:18:02,096 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 6,170,000 documents indexed
2024-02-12 19:19:02,099 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 6,230,000 documents indexed
2024-02-12 19:20:02,102 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 6,290,000 documents indexed
2024-02-12 19:21:02,105 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 6,340,000 documents indexed
2024-02-12 19:22:02,108 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 6,400,000 documents indexed
2024-02-12 19:23:02,111 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 6,450,000 documents indexed
2024-02-12 19:24:02,113 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 6,480,000 documents indexed
2024-02-12 19:25:02,115 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 6,520,000 documents indexed
2024-02-12 19:26:02,119 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 6,580,000 documents indexed
2024-02-12 19:27:02,122 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 6,630,000 documents indexed
2024-02-12 19:28:02,126 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 6,690,000 documents indexed
2024-02-12 19:29:02,129 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 6,750,000 documents indexed
2024-02-12 19:30:02,132 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 6,810,000 documents indexed
2024-02-12 19:31:02,135 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 6,870,000 documents indexed
2024-02-12 19:32:02,139 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 6,930,000 documents indexed
2024-02-12 19:33:02,142 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 6,990,000 documents indexed
2024-02-12 19:34:02,146 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 7,040,000 documents indexed
2024-02-12 19:35:02,149 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 7,100,000 documents indexed
2024-02-12 19:36:02,153 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 7,160,000 documents indexed
2024-02-12 19:37:02,156 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 7,220,000 documents indexed
2024-02-12 19:38:02,160 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 7,280,000 documents indexed
2024-02-12 19:39:02,162 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 7,340,000 documents indexed
2024-02-12 19:40:02,166 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 7,400,000 documents indexed
2024-02-12 19:41:02,169 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 7,460,000 documents indexed
2024-02-12 19:42:02,173 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 7,520,000 documents indexed
2024-02-12 19:43:02,176 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 7,580,000 documents indexed
2024-02-12 19:44:02,181 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 7,640,000 documents indexed
2024-02-12 19:45:02,184 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 7,710,000 documents indexed
2024-02-12 19:46:02,188 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 7,770,000 documents indexed
2024-02-12 19:47:02,191 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 7,830,000 documents indexed
2024-02-12 19:48:02,195 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 7,890,000 documents indexed
2024-02-12 19:49:02,198 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 7,960,000 documents indexed
2024-02-12 19:50:02,201 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 8,020,000 documents indexed
2024-02-12 19:51:02,205 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 8,080,000 documents indexed
2024-02-12 19:52:02,209 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 8,150,000 documents indexed
2024-02-12 19:53:02,213 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 8,210,000 documents indexed
2024-02-12 19:54:02,216 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 8,280,000 documents indexed
2024-02-12 19:55:02,219 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 8,340,000 documents indexed
2024-02-12 19:56:02,223 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 8,410,000 documents indexed
2024-02-12 19:57:02,226 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 8,480,000 documents indexed
2024-02-12 19:58:02,230 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 8,550,000 documents indexed
2024-02-12 19:59:02,234 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 8,630,000 documents indexed
2024-02-12 20:00:02,238 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 8,700,000 documents indexed
2024-02-12 20:01:02,241 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:258) - 8,790,000 documents indexed
2024-02-12 20:01:46,669 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:306) - Indexing Complete! 8,841,823 documents indexed
2024-02-12 20:01:46,670 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:307) - ============ Final Counter Values ============
2024-02-12 20:01:46,670 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:308) - indexed:        8,841,823
2024-02-12 20:01:46,670 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:309) - unindexable:            0
2024-02-12 20:01:46,670 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:310) - empty:                  0
2024-02-12 20:01:46,671 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:311) - skipped:                0
2024-02-12 20:01:46,671 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:312) - errors:                 0
2024-02-12 20:01:46,677 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:315) - Total 8,841,823 documents indexed in 02:41:44
2024-02-12 20:01:50,080 INFO  [main] search.SearchCollection (SearchCollection.java:990) - ============ Initializing Searcher ============
2024-02-12 20:01:50,083 INFO  [main] search.SearchCollection (SearchCollection.java:991) - Index: ./data/splade/english-marco/train/distill/checkpoint-157165/evaluate/corpus
2024-02-12 20:01:50,248 INFO  [main] search.SearchCollection (SearchCollection.java:994) - Fields: []
2024-02-12 20:01:51,290 INFO  [main] search.SearchCollection (SearchCollection.java:1317) - ============ Launching Search Threads ============
2024-02-12 20:01:51,291 INFO  [main] search.SearchCollection (SearchCollection.java:1318) - runtag: Anserini
2024-02-12 20:01:57,801 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 100 queries processed
2024-02-12 20:02:01,545 INFO  [pool-3-thread-4] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 200 queries processed
2024-02-12 20:02:05,772 INFO  [pool-3-thread-2] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 300 queries processed
2024-02-12 20:02:09,791 INFO  [pool-3-thread-4] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 400 queries processed
2024-02-12 20:02:15,020 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 500 queries processed
2024-02-12 20:02:20,542 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 600 queries processed
2024-02-12 20:02:25,272 INFO  [pool-3-thread-1] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 700 queries processed
2024-02-12 20:02:29,642 INFO  [pool-3-thread-4] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 800 queries processed
2024-02-12 20:02:34,226 INFO  [pool-3-thread-2] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 900 queries processed
2024-02-12 20:02:39,654 INFO  [pool-3-thread-2] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 1000 queries processed
2024-02-12 20:02:43,823 INFO  [pool-3-thread-1] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 1100 queries processed
2024-02-12 20:02:48,082 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 1200 queries processed
2024-02-12 20:02:51,863 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 1300 queries processed
2024-02-12 20:02:57,776 INFO  [pool-3-thread-1] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 1400 queries processed
2024-02-12 20:03:01,805 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 1500 queries processed
2024-02-12 20:03:05,872 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 1600 queries processed
2024-02-12 20:03:10,262 INFO  [pool-3-thread-2] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 1700 queries processed
2024-02-12 20:03:14,506 INFO  [pool-3-thread-4] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 1800 queries processed
2024-02-12 20:03:18,411 INFO  [pool-3-thread-2] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 1900 queries processed
2024-02-12 20:03:22,331 INFO  [pool-3-thread-1] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 2000 queries processed
2024-02-12 20:03:25,767 INFO  [pool-3-thread-2] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 2100 queries processed
2024-02-12 20:03:30,690 INFO  [pool-3-thread-4] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 2200 queries processed
2024-02-12 20:03:35,364 INFO  [pool-3-thread-1] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 2300 queries processed
2024-02-12 20:03:37,984 INFO  [pool-3-thread-4] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 2400 queries processed
2024-02-12 20:03:42,680 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 2500 queries processed
2024-02-12 20:03:47,859 INFO  [pool-3-thread-1] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 2600 queries processed
2024-02-12 20:03:52,795 INFO  [pool-3-thread-4] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 2700 queries processed
2024-02-12 20:03:58,299 INFO  [pool-3-thread-2] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 2800 queries processed
2024-02-12 20:04:02,966 INFO  [pool-3-thread-2] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 2900 queries processed
2024-02-12 20:04:07,778 INFO  [pool-3-thread-4] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 3000 queries processed
2024-02-12 20:04:12,553 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 3100 queries processed
2024-02-12 20:04:16,942 INFO  [pool-3-thread-4] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 3200 queries processed
2024-02-12 20:04:21,871 INFO  [pool-3-thread-1] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 3300 queries processed
2024-02-12 20:04:26,693 INFO  [pool-3-thread-4] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 3400 queries processed
2024-02-12 20:04:31,108 INFO  [pool-3-thread-1] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 3500 queries processed
2024-02-12 20:04:35,960 INFO  [pool-3-thread-1] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 3600 queries processed
2024-02-12 20:04:40,992 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 3700 queries processed
2024-02-12 20:04:46,288 INFO  [pool-3-thread-2] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 3800 queries processed
2024-02-12 20:04:51,841 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 3900 queries processed
2024-02-12 20:04:56,910 INFO  [pool-3-thread-1] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 4000 queries processed
2024-02-12 20:05:01,658 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 4100 queries processed
2024-02-12 20:05:05,783 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 4200 queries processed
2024-02-12 20:05:10,165 INFO  [pool-3-thread-2] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 4300 queries processed
2024-02-12 20:05:14,696 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 4400 queries processed
2024-02-12 20:05:19,644 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 4500 queries processed
2024-02-12 20:05:25,357 INFO  [pool-3-thread-2] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 4600 queries processed
2024-02-12 20:05:30,981 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 4700 queries processed
2024-02-12 20:05:36,204 INFO  [pool-3-thread-4] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 4800 queries processed
2024-02-12 20:05:40,477 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 4900 queries processed
2024-02-12 20:05:45,221 INFO  [pool-3-thread-4] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 5000 queries processed
2024-02-12 20:05:49,812 INFO  [pool-3-thread-2] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 5100 queries processed
2024-02-12 20:05:54,503 INFO  [pool-3-thread-2] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 5200 queries processed
2024-02-12 20:05:59,384 INFO  [pool-3-thread-2] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 5300 queries processed
2024-02-12 20:06:04,537 INFO  [pool-3-thread-4] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 5400 queries processed
2024-02-12 20:06:10,612 INFO  [pool-3-thread-1] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 5500 queries processed
2024-02-12 20:06:15,852 INFO  [pool-3-thread-2] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 5600 queries processed
2024-02-12 20:06:21,338 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 5700 queries processed
2024-02-12 20:06:25,775 INFO  [pool-3-thread-1] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 5800 queries processed
2024-02-12 20:06:30,865 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 5900 queries processed
2024-02-12 20:06:34,341 INFO  [pool-3-thread-1] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 6000 queries processed
2024-02-12 20:06:38,885 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 6100 queries processed
2024-02-12 20:06:43,917 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 6200 queries processed
2024-02-12 20:06:48,307 INFO  [pool-3-thread-4] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 6300 queries processed
2024-02-12 20:06:53,400 INFO  [pool-3-thread-4] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 6400 queries processed
2024-02-12 20:06:58,268 INFO  [pool-3-thread-2] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 6500 queries processed
2024-02-12 20:07:02,976 INFO  [pool-3-thread-4] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 6600 queries processed
2024-02-12 20:07:07,446 INFO  [pool-3-thread-4] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 6700 queries processed
2024-02-12 20:07:13,014 INFO  [pool-3-thread-3] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 6800 queries processed
2024-02-12 20:07:17,948 INFO  [pool-3-thread-2] search.SearchCollection$SearcherThread (SearchCollection.java:893) - ranker: impact(), reranker: default: 6900 queries processed
2024-02-12 20:07:22,365 INFO  [pool-2-thread-1] search.SearchCollection$SearcherThread (SearchCollection.java:914) - ranker: impact(), reranker: default: 6980 queries processed 
in 00:05:31 = ~21.09 q/s
2024-02-12 20:07:41,473 INFO  [main] search.SearchCollection (SearchCollection.java:1390) - Total run time: 00:05:51
{'NDCG@1': 0.24585, 'NDCG@3': 0.35683, 'NDCG@5': 0.39849, 'NDCG@10': 0.43615, 'NDCG@100': 0.48968, 'NDCG@1000': 0.50018}
{'MAP@1': 0.23915, 'MAP@3': 0.32676, 'MAP@5': 0.35028, 'MAP@10': 0.36618, 'MAP@100': 0.37739, 'MAP@1000': 0.37781}
{'Recall@10': 0.6507, 'Recall@50': 0.84254, 'Recall@100': 0.89741, 'Recall@200': 0.93381, 'Recall@500': 0.96544, 'Recall@1000': 0.97744}
{'P@1': 0.24585, 'P@3': 0.1511, 'P@5': 0.11172, 'P@10': 0.06799, 'P@100': 0.00949, 'P@1000': 0.00104}
{'MRR@10': 0.37226, 'MRR@100': 0.38281, 'MRR@1000': 0.38319}