Bulksearch
Lightweight and read-write optimized full text search library.
Install / Use
npx skills add nextapps-de/bulksearchInstalls into whichever agent you are using.
README
BulkSearch
Lightweight and read-write optimized full text search library.
When it comes to the overall speed, BulkSearch outperforms every searching library out there and also provides flexible search capabilities like multi-word matching, phonetic transformations or partial matching. It is essentially based on how a HDD manages files in a filesystem. Adding, updating or removing items are as fast as searching for them, but also consumes some additional memory. When your index doesn't need to be updated frequently then <a href="flexsearch/" target="_blank">FlexSearch</a> may be a better choice. BulkSearch also provides you a asynchronous processing model to perform queries in the background.
Benchmark:
- Comparison: <a href="https://jsperf.com/compare-search-libraries" target="_blank">https://jsperf.com/compare-search-libraries</a>
- Detailed: <a href="https://jsperf.com/bulksearch" target="_blank">https://jsperf.com/bulksearch</a>
Supported Platforms:
- Browser
- Node.js
Supported Module Definitions:
- AMD (RequireJS)
- CommonJS (Node.js)
- Closure (Xone)
- Global (Browser)
All Features:
<ul> <li>Partial Words</li> <li>Multiple Words</li> <li>Flexible Word Order</li> <li><a href="#phonetic">Phonetic Search</a></li> <li>Limit Results</li> <li><a href="#pagination">Pagination</a></li> <li>Caching</li> <li>Asynchronous Mode</li> <li>Custom Matchers</li> <li>Custom Encoders</li> </ul>Installation
HTML / Javascript
<html>
<head>
<script src="js/bulksearch.min.js"></script>
</head>
...
Note: Use bulksearch.min.js for production and bulksearch.js for development.
Use latest from CDN:
<script src="https://cdn.rawgit.com/nextapps-de/bulksearch/master/bulksearch.min.js"></script>
Node.js
npm install bulksearch
In your code include as follows:
var BulkSearch = require("bulksearch");
Or pass in options when requiring:
var index = require("bulksearch").create({/* options */});
AMD
var BulkSearch = require("./bulksearch.js");
Compare BulkSearch vs. FlexSearch
<table> <tr></tr> <tr> <th align="left">Description</th> <th align="left">BulkSearch</th> <th align="left">FlexSearch</th> </tr> <tr> <td>Access</td> <td>Read-Write optimized index</td> <td>Read-Memory optimized index</td> </tr> <tr></tr> <tr> <td>Memory</td> <td>Large (~ 90 bytes per word)</td> <td>Tiny (~ 2 bytes per word)</td> </tr> <tr></tr> <tr> <td>Usage</td> <td><ul><li>Limited content</li><li>Index updates continously</li></ul></td> <td><ul><li>Fastest possible search</li><li>Rare updates on index</li><li>Low memory capabilities</li></ul></td> </tr> <tr></tr> <tr> <td>Limit Results</td> <td>Yes</td> <td>Yes</td> </tr> <tr></tr> <tr> <td>Pagination</td> <td>Yes</td> <td>No</td> </tr> </table>API Overview
Global methods:
- <a href="#bulksearch.create">BulkSearch.create(<options>)</a>
- <a href="#bulksearch.addmatcher">BulkSearch.addMatcher({KEY: VALUE})</a>
- <a href="#bulksearch.register">BulkSearch.register(name, encoder)</a>
- <a href="#bulksearch.encode">BulkSearch.encode(name, string)</a>
Index methods:
- <a href="#index.add">Index.add(id, string)</a>
- <a href="#index.search1">Index.search(string, <limit>, <callback>)</a>
- <a href="#index.search2">Index.search(string, <page>, <callback>)</a>
- <a href="#index.search3">Index.search(options, <callback>)</a>
- <a href="#index.update">Index.update(id, string)</a>
- <a href="#index.remove">Index.remove(id)</a>
- <a href="#index.reset">Index.reset()</a>
- <a href="#index.destroy">Index.destroy()</a>
- <a href="#index.init">Index.init(<options>)</a>
- <a href="#index.optimize">Index.optimize()</a>
- <a href="#index.info">Index.info()</a>
- <a href="#index.addmatcher">Index.addMatcher({KEY: VALUE})</a>
- <a href="#index.encode">Index.encode(string)</a>
Usage
<a name="bulksearch.create"></a>
Create Index
BulkSearch.create(<options>)
var index = new BulkSearch();
alternatively you can also use:
var index = BulkSearch.create();
Create index with custom options
var index = new BulkSearch({
// default values:
type: "integer",
encode: "icase",
boolean: "and",
size: 4000,
multi: false,
strict: false,
ordered: false,
paging: false,
async: false,
cache: false
});
Read more: <a href="#phonetic">Phonetic Search</a>, <a href="#compare">Phonetic Comparison</a>, <a href="#memory">Improve Memory Usage</a> <a name="index.add"></a>
Add items to an index
Index.add(id, string)
index.add(10025, "John Doe");
<a name="index.search1"></a>
Search items
Index.search(string|options, <limit|page>, <callback>)
index.search("John");
Limit the result:
index.search("John", 10);
Perform queries asynchronously:
index.search("John", function(result){
// array of results
});
<a name="index.search3"></a> Pass parameter as an object:
index.search({
query: "John",
page: '1:1234',
limit: 10,
callback: function(result){
// async
}
});
<a name="index.update"></a>
Update item from an index
Index.update(id, string)
index.update(10025, "Road Runner");
<a name="index.remove"></a>
Remove item from an index
Index.remove(id)
index.remove(10025);
<a name="index.reset"></a>
Reset index
index.reset();
<a name="index.destroy"></a>
Destroy index
index.destroy();
<a name="index.init"></a>
Re-Initialize index
Index.init(<options>)
Note: Re-initialization will also destroy the old index!
Initialize (with same options):
index.init();
Initialize with new options:
index.init({
/* options */
});
<a name="bulksearch.addmatcher"></a>
Add custom matcher
BulkSearch.addMatcher({REGEX: REPLACE})
Add global matchers for all instances:
BulkSearch.addMatcher({
'ä': 'a', // replaces all 'ä' to 'a'
'ó': 'o',
'[ûúù]': 'u' // replaces multiple
});
<a name="index.addmatcher"></a> Add private matchers for a specific instance:
index.addMatcher({
'ä': 'a', // replaces all 'ä' to 'a'
'ó': 'o',
'[ûúù]': 'u' // replaces multiple
});
Add custom encoder
Define a private custom encoder during creation/initialization:
var index = new BulkSearch({
encode: function(str){
// do something with str ...
return str;
}
});
<a name="bulksearch.register"></a>
Register a global encoder to be used by all instances
BulkSearch.register(name, encoder)
BulkSearch.register('whitespace', function(str){
return str.replace(/ /g, '');
});
Use global encoders:
var index = new BulkSearch({ encode: 'whitespace' });
<a name="index.encode"></a>
Call encoders directly
Private encoder:
var encoded = index.encode("sample text");
<a name="bulksearch.encode"></a> Global encoder:
var encoded = BulkSearch.encode("whitespace", "sample text");
Mixup/Extend multiple encoders
BulkSearch.register('mixed', function(str){
str = this.encode("icase", str); // built-in
str = this.encode("whitespace", str); // custom
return str;
});
BulkSearch.register('extended', function(str){
str = this.encode("custom", str);
// do something additional with str ...
return str;
});
<a name="index.info"></a>
Get info
index.info();
Returns information about the index, e.g.:
{
"bytes": 103600,
"chunks": 9,
"fragmentation": 0,
"fragments": 0,
"id": 0,
"length": 7798,
"matchers": 0,
"size": 10000,
"status": false
}
Note: When the fragmentation value is about 50% or higher, your should consider using cleanup(). <a name="index.optimize"></a>
Optimize / Cleanup index
Optimize an index will free all fragmented memory and also rebuilds the index by scoring.
index.optimize();
<a name="pagination"></a>
Pagination
Note: Pagination can simply reduce query time by a factor of 100.
Enable pagination on initialization:
var index = BulkSearch.create({ paging: true });
Perform query and pass a limit (items per page):
index.search("John", 10);
The response will include a pagination object like this:
{
"current": "0:0",
"prev": null,
"next": "1:16322",
"results": []
}
Explanation:
<table> <tr> <td align="left">"current"</th> <td align="left">Includes the pointer to the current page.</th> </tr> <tr></tr> <tr> <td align="left">"prev"</th> <td align="left">Includes the pointer to the previous page. Whenever this field has the value <i>null</i> there are no more previous pages available.</th> </tr> <tr></tr> <tr> <td align="left">"next"</th> <td align="left">Includes the pointer to the next page. Whenever this field has the value <i>null</i> there are no more pages left.</th> </tr> <tr></tr> <tr> <td align="left">"results"</th> <td align="left">Array of matched items.</th> </tr> </table> <a name="index.search2"></a>Perform query and pass a pointer to a specific page:
index.search("John", {
page: "1:16322", // pointer
limit: 10
});
<a name="options" id="options"></a>
Options
<table> <tr> <th align="left">Option</th> <th align="left">Values</th> <th align="left"Related Skills
node-connect
385.5kDiagnose OpenClaw Android, iOS, or macOS node pairing, QR/setup code, route, auth, and connection failures.
blender-python-addon
40.5kBlender Python add-on rules for operators, panels, properties, registration, testing, and API-safe scripting
flutter-development-guidelines-cursorrules-prompt-file
40.5kCursor rules for Flutter development with MVVM architecture, Riverpod state management, Material widgets, and Dart style guidelines.
commit-push-pr
140.7kCommit, push, and open a PR
