Clustering Search Engine Based on Independent Users
Abstract
The rapid development of the Internet has provided more and more internet information. In order to search the required information rapidly, search engines have become one of the indispensable internet tools. Nowadays, although many search engine systems have been applying many methods to improve the retrieval precision, the retrieved results still include a lot of irrelevance documents mixing with the relevance ones, and it brings users an additional burden.
Based on the profile of search engines and cluster analysis, the clustering search engine for the independent users is designed. It helps the web users choose required information from along list of returned snippets, clustering of the retrieved results will assign snippets to automatically groups based on calculating the similarity. The groups (clusters) formed should have a high degree of association between members of the same groups and a low degree between members of different groups. So the users can only view their interested groups and save much time. In the process of design and implementation, when a independent search request is send to the search engine, the Yahoo API is used to obtain the necessary research data sources in our system and inverted files indexing model is used to create index while using the key words denoting index. The similarity is calculated according to the title of retuned results, URL and summary in our system, then search results and their relationship of similarity are mapped to an undirected graph. Finally, the points of the undirected map are clustered according to the similarity of each point in the undirected map and the ultimate results are gotten.
During the clustering process, a new clustering method is proposed. Firstly, the method randomly selects a few points as initial centers. Secondly, each point is gradually added one or several clusters based on re-calculation of the similarity between added point and each center. If the similarity is greater than predefined threshold, the point will be added some cluster, whose center are adjusted until adding all the points. Theoretic analysis shows that the used reverted file model in our system needs little resources and proposed clustering algorithm resolves the problem of . And the isolating point problem is also considered. Experimental results also show that the proposed method has better clustering performance and the returned results can be clustered effectively.