SEQuRAmA
I've continued tweaking and adding search within search features to SEQuRAmA--the Search Engine QUery Result AccuMulator Aggregator. But in doing so I organized the various features under five possible categories.
Five Categories
The five categories are:
- verification
- presentation
- operation
- optimization
- improvements
List of Features by Category
Some of the features I have implemented, others are on the "to-do" list of software improvements. But by organizing the features into categories, it is easier to prioritize. Some features lead to other features, implementing one facilitates another implementation, or more simply without one feature already working, another cannot be so easily implemented. (Never say the word
impossible in software, famous last words of many that uttered the word...)
The Features of SEQuRAmA
Verification:
- Verify link media type (xml, pdf, html)
- Verify link exists
Presentation:
- Color code link for reliability (exists, unknown)
- Cluster links together by type (xml, pdf, html, gif)
- Cluster similar links within domain
- Internal link to each cluster for easy navigation
operation:
- Cookie control from search engines (delete, store)
- No search engine advertisements in output results
- Query multiple search engines in tandem
Optimization:
- Permutations on search keywords--"Richard M. Nixon" one variant "Richard Nixon" and "Nixon"
- Rank resultant link by commonality--search 12-search engines 9/12 = 0.75 rank
- Connect private, internal resource (such internal employee website portal) to external publicly accessible resource
Improvements:
- For non-existent link, replace with "The Wayback Machine" link if available, or search engine cache, or both
- Using most ranked results, tweak search using content in web pages (for example ID3 algorithm to classify web page)
- Determine other search keywords from words/markup on web pages (from Nixon, get Watergate, trip to China)
- Store advertisements returned as separate results accessed outside primary results
- Use a presentation template that specifies how to organize and structure the results
Modular, Multi-threaded, Multi-Class
One important implementation and design consideration is avoiding a monolithic block of Java source code. Each search is its own class, and a thread, implementing an interface, so that each search engine is a module. One change I've considered is dynamically loading a bytecode .class file, and then unloading if the search engine is down, or unresponsive within a wall clock timed time threshold.
Other functionality accesses the raw results, stored in an ordered map (or multi-map since the resultant datum is stored within nested ordered maps). The original "store" (using a term from Babbage's Analytic Engine) used the standard Java data structures, but for more performance (which creeps up as I had more search engine modules...) I used open-source code, and implemented with a more specialized interface for specific-functionality and not general-purpose operation.
Do It Yourself
I implemented or created SEQuRAmA as a mix of fun, challenge, and a much useful tool for more efficient search online. Later one possibility is to have SEQuRAma work as a database of results, searchable through a web browser. Turing a super search engine into a local database, perhaps even extracting keywords from the query (and using the search terms that led to the resulting link) and result to organize the data internally. Of course, every software engineer often suffers from "creeping featuritis" and I'll have to reclassify these possible enhancements, but before then continue to tweak and improve the features already on the "to-do" list.
Other features are superfluous...such as quote of the day, important historical events by date, jokes from search results...nice, but add nothing to efficient search across multiple search engines.
Seek, locate, accumulate (a pun and paraphrase of the Daleks from Dr. Who...)
