Homepage - https://github.com/FabriceColin/pinot previously hosted at http://code.google.com/p/pinot-search/ and http://pinot.berlios.de/ Translations - https://translations.launchpad.net/pinot/trunk/+pots/pinot

What is Pinot
Available engines
Indexes
Indexing and monitoring
Searching
Viewing cached results
File formats
File patterns
Digging deeper
Saving results
D-Bus service & daemon
CJKV support
Environment variables and aliases
How to reset indexes
Compiling
What is Pinot

Pinot combines desktop search and metasearch. It consists of : * a D-Bus service daemon that crawls, indexes, monitors your documents and that plugs into the GNOME Shell search system ("pinot-dbus-daemon") * a GTK3-based user interface that enables to query the index built by the service as well as Web engines, and which can display and analyze the results ("pinot") * other command-line tools

It was developed and tested on GNU/Linux and should work on other Unix-like systems.

Available engines

One of the main functionalities of Pinot is metasearch. This lets you query a variety of sources, including Web-based search engines. By default, the list of available engines is hidden and defaults to internal indexes (see section "3. Indexes"). To show the list of engines, click on the Show All Search Engines button, next to the Query field immediately below the menu bar. Click on the same button again to hide the list.

Any number of engine or engine group may be selected at any one time. Multi-selection is done like in any other application. All queries are always run against the list of currently selected engines.

Pinot supports both Sherlock and OpenSearch Description plugins. They are installed in $PREFIX/share/pinot/engines/, where PREFIX is usually /usr. Additional engines can be installed in that directory or in ~/.pinot/engines. Note this directory is not created automatically.

Sherlock is what Firefox and the Mozilla Suite use. Chances are that somebody wrote a plugin for the engine you are interested in. Beware that a lot are out of date and will require some changes. Use pinot-search on the command-line to run a quick check on a plugin, eg $ pinot-search sherlock $PREFIX/share/pinot/engines/Bozo.src "clowns"

Plugins are categorized by channels. For Sherlock plugins, the routeType element under SEARCH specifies the name of the channel the plugin belongs to.

As for OpenSearch, Pinot should work with OpenSearch Description 1.0 and 1.1 (draft 2) plugins. Keep in mind that the spec doesn't describe how to parse the results pages returned by search engines, therefore Pinot assumes that engines return results formatted according to the OpenSearch Response standard. In practice, this means that plugins that don't stick to the following rules will be ignored or won't show any result : * For Description 1.1 plugins, the type attribute on the Url field must be set to "application/atom+xml" or "application/rss+xml" (default). "text/html" will be rejected. * The search engine's results page content type must be some form of XML, otherwise Pinot won't attempt parsing it. Pinot differs from the Description spec in that it interprets the Tags field as a channel name. The standard defines Tags as a "space-delimited set of words that are used as keywords to identify and categorize this search content".

The "Xapian Omega" plugin allows to query a locally installed instance of Xapian Omega at http://localhost/. If Omega is installed elsewhere, edit $PREFIX/share/pinot/engines/OmegaDescription.xml.

Indexes

Pinot has two internal indexes. My Documents is populated by the D-Bus service and contains documents found on your computer. My Web Pages is populated by the UI whenever you : * import an external document, using the Index, Import URL menu * index results returned by Web engines, using the Results, Index menu or through a Stored Query Both index may have any of the file types listed in section "7. File formats".

Indexes built by any other Xapian-based tools can be added to Pinot. To add an external index, click the + button at the bottom of the engines list. It can either be local, in which case you will have to select the directory where it is found, or served from a remote machine by xapian-tcpsrv. See the manual page for xapian-tcpsrv(1).

All indexes are grouped together under the channel Current User in the engines list.

Indexing and monitoring

Pinot can index any directory configured under the Indexing tab of the Preferences box. Monitoring is optional and should be disabled for the directories whose contents seldom change, eg $PREFIX/share/doc. Indexing and monitoring of directories is handled by the D-Bus service. The number of files and directories that can be monitored is capped by the value of /proc/sys/fs/inotify/max_user_watches - 1024.

Symlinks are not followed but are still indexed, with the MIME type "inode/symlink".

While Pinot is not currently able to get to and index application-specific data held in dot-directories, it can index common file formats as listed in section "7. File formats".

All files and directories with a name that starts with a dot, eg ".thunderbird", are skipped and their content is not indexed. If you wish to include the contents of some dot-directory, create a symlink to a directory that is configured in Preferences. For instance, if "~/Documents" is configured for indexing, create a symlink from "~/.thunderbird" to "~/Documents/TMail". For this to work, the dot-directory must not be in a directory configured for indexing.

If you want to exclude any specific files or directories from indexing, use patterns as described in section "8. File patterns".

Pinot supports stopwords removal. While no such list is provided by default, they can be easily found on the Internet. Each language has its own stopword list, for instance a stopwords list for English should be copied to $PREFIX/share/pinot/stopwords/stopwords.en

Language detection is done with libexttextcat. Ensure that the paths listed in /etc/pinot/textcat_conf.txt are correct.

The pinot-index program allows indexing and peeking at documents' properties from the command-line. Using the -i/--index option with the My Documents or My Web Pages index is not recommended. For more details, see the manual page for pinot-index(1).

Searching

Searches are run differently based on the type of engine being queried.

When querying a Web engine, Pinot assumes this engine understands the query, which is sent as is. No pre-processing is performed on the text of the query, and the results list is more or less presented as retrieved from the Web engine.

When querying an index, things are somewhat different. Queries can be expressed in a very natural way, using a combination of operators, filters and ranges. This query syntax is the syntax supported natively by Xapian's QueryParser and is documented at http://www.xapian.org/docs/queryparser.html For instance, the query "type:text/html AND lang:en AND (tcp NEAR ip)" will look for HTML files in English that mention TCP/IP. Note that all operators should be specified in capitals, eg "AND" not "and". The latter will be treated as a regular term.

Pinot supports these query filters : "site" for host name, eg "site:github.com" "file" for file name, eg "file:index.html" "ext" for file extension, eg "ext:html" "title" for title, eg "title:pinot" "url" for URL, eg "url:https://github.com/" "dir" for directory, eg "dir:/home/fabrice" "inurl" for documents embedded in a URL, eg "inurl:file:///home/fabrice/Documents/backup.tar.gz" "lang" for ISO language code, eg "lang:en" "type" for MIME type, eg "type:text/html" "class" for MIME type classification, eg "class:text" "label" for label, eg "label:Important"

The directory filter is recursive, ie it applies to sub-directories. Allowed language codes are "da", "nl", "en", "fi", "fr", "de", "hu", "it", "nn", "pt", "ro", "ru", "es", "sv" and "tr".

Stemming is available to stored queries for which a stemming language is defined. If such a query doesn't return any exact match, the query terms are stemmed and the query is run again. Stopwords are also then removed if a stopwords list was found for the stemming language.

The values of "file", "url", "dir" and "label" may be double-quoted. It's also worth pointing out that the query "dir:/X/Y" will return files and directories located in /X/Y, but not Y itself, which is what "dir:/X file:Y" would do.

In addition, these ranges are supported : "YYYYMMDD..YYYYMMDD" for date ranges, eg "20070801..20070831" "HHMMSS..HHMMSS" for time ranges, eg "090000..180000" "size0..size1b" for size in bytes, eg "0..10240b"

See the manual page for pinot-search(1) for examples.

Viewing cached results

Results returned by search engines can be viewed "live" by selecting the View menuitem under Results. This opens whatever application defined for the result's MIME type and/or protocol scheme. In addition, Pinot allows to view the page as cached by Google and the Wayback Machine. Cache providers are actually configured in globalconfig.xml, located in /etc/pinot/. For instance : <cache> <name>Google</name> <location>http://www.google.com/search?q=cache:%url0</location> <protocols>http, https</protocols> </cache>

This is self-explanatory :-) Here it configures a cache provider called "Google" that handles both http and https. The location field supports two parameters that are substituted to obtain th

Pinot

Install / Use

README