Figure 5: Main screens for (a) searching by selecting from several subjects or by text, (b) detailed subject navigation.
To search for images and videos, the user issues a query, which extracts items from the catalog. The user-interface and main search screens are illustrated in Figure 5(a) and (b). The user may initiate the search by entering terms or by selecting subjects directly. The overall search process and model for user-interaction is depicted in Figure 6. As illustrated, a query for images and videos produces a search results list, list A, which is presented to the user. For example, Figure 7(a) illustrates the search results list for a query for images and videos related to ``nature'', that is, 
. The user may manipulate, search or view A.
Figure 6: Search, retrieval and search results list manipulation processes.
After possibly searching and/or viewing the search results list, it is fed-back to the manipulation module as list C, as illustrated in Figure 6. The user manipulates C by adding or removing records. This is done by issuing a new query that creates a second, intermediate list A. The user then generates a new search results list B by selecting one of the following manipulations on C using A, for example, define 
and 
, then
The user may browse and search the list B using both content-based and text-based tools. In the case of content-based searching, the output is list C, where 
gives an ordered subset of B, and C is ordered by highest similarity to the user's selected item. In the current system, list C is truncated to 
records, where 
may be adjusted by the user. The search and browse operations may be conducted on the input list B or on the entire catalog at any time. In the first case, the user may browse the current search results list by selecting one of the items and instructing the system to reorder the list by highest similarity to the selected item.
For example, 
, where 
means visual similarity, ranks list B in order of highest similarity to a selected item from B. For example, the following content-based visual query:
ranks the ``nature'' images and videos in order of highest visual similarity to the selected ``mountain scene image.'' Alternatively, the user can select one of the items in the current search results list B to search the entire catalog for similar items. For example, 
ranks list A, where A is the full catalog, in order of highest visual similarity to the selected item from B. In the example illustrated in Figure 7(b), the query 
, retrieves the images and videos from the full catalog that are most similar to the selected image of a ``red race car.''
Figure 7: (a) Search results for SUBJECT = ``nature'', (b) content-based visual query results for images/videos 
.
The user has several options for viewing search results. Since the visual information requires more communication bandwidth than text, the user is given control in viewing and browsing the search results to enable them to be inspected quickly. The default view presents for each catalog record a small (approximately 
pixels) icon for each image and video scene in addition to other relevant fields, see Figure 7(a) and (b). Alternatively, the user can select to eliminate the display of the icon altogether, in which case the name of the image/video is displayed. Only 
records at a time are presented to the user in the view. The system gives the user controls to navigate the list by getting the next, previous and top 
records. The user may conveniently select an item for full download, which retrieves the image/video from the original URL to the user.