Voice & Image Search

The widget allows searching for products using voice or images, providing a multimodal search experience.

Voice Search

Activation

Voice search is enabled by default:

NeuroonWidget.init({
  features: {
    voiceSearch: true, // Default: true
  },
});

Requirements

Requirement	Detail
HTTPS	Required (except localhost)
Browsers	Chrome, Safari, Edge
Permissions	Microphone access

Supported languages

Language	Code	Accuracy
Spanish	es-ES	High
English	en-US	High

Language is automatically detected based on the widget's locale setting.

How it works

1. User clicks the microphone icon
2. Microphone permission is requested (first time)
3. Widget shows "Listening..."
4. User dictates their search
5. Speech is transcribed and search executed

// Voice states
widget.on('voice:start', () => {
  console.log('Microphone active');
});

widget.on('voice:end', ({ transcript }) => {
  console.log('Transcript:', transcript);
});

widget.on('voice:error', ({ error }) => {
  console.error('Voice error:', error.message);
});

Error handling

type VoiceError =
  | 'not-allowed'      // Permission denied
  | 'no-speech'        // No speech detected
  | 'aborted'          // User cancelled
  | 'network'          // Network error
  | 'not-supported';   // Browser not supported

widget.on('voice:error', ({ error }) => {
  switch (error.code) {
    case 'not-allowed':
      showNotification('Please allow microphone access in your browser');
      break;
    case 'no-speech':
      showNotification("I didn't hear you. Try again");
      break;
    case 'not-supported':
      showNotification('Your browser does not support voice search');
      break;
  }
});

---

Image Search

Activation

Image search is enabled by default:

NeuroonWidget.init({
  features: {
    imageSearch: true, // Default: true
  },
});

Supported formats

Format	Extension	Support
JPEG	.jpg, .jpeg	Full
PNG	.png	Full
WebP	.webp	Full
GIF	.gif	First frame only
HEIC	.heic	Auto-conversion

Limits

Parameter	Value
Maximum size	5 MB
Minimum resolution	100x100 px
Maximum resolution	4096x4096 px

Input methods

1. Upload file: Click camera icon → Select file
2. Drag and drop: Drag & drop image onto widget
3. Camera capture: On mobile, option to take photo

How it works

1. User uploads/captures an image
2. Image is resized if needed (max 1024px)
3. Sent to backend for analysis
4. Engine generates image embeddings (OpenCLIP)
5. Visually similar products are searched
6. Results sorted by visual similarity

widget.on('image:upload', ({ file }) => {
  console.log('Image uploaded:', file.name, file.size);
});

widget.on('image:result', ({ products }) => {
  console.log('Similar products:', products.length);
});

Image search API

// Upload image programmatically
const file = document.getElementById('my-file-input').files[0];
await widget.searchByImage(file);

// With image URL (CORS allowed)
await widget.searchByImageUrl('https://example.com/product.jpg');

Image events

Event	Payload	Description
image:upload	{ file }	Image selected
image:processing	{}	Processing image
image:result	{ products }	Results received
image:error	{ error }	Search error

widget.on('image:processing', () => {
  showLoadingSpinner();
});

widget.on('image:result', ({ products }) => {
  hideLoadingSpinner();
  if (products.length === 0) {
    showNotification('No similar products found');
  }
});

Image error handling

type ImageError =
  | 'invalid-format'   // Unsupported format
  | 'file-too-large'   // Exceeds 5MB
  | 'resolution-low'   // Less than 100x100
  | 'processing-error' // Server error
  | 'network';         // Network error

widget.on('image:error', ({ error }) => {
  switch (error.code) {
    case 'invalid-format':
      showNotification('Use JPG, PNG, or WebP');
      break;
    case 'file-too-large':
      showNotification('Image too large (max 5MB)');
      break;
    case 'resolution-low':
      showNotification('Image too small');
      break;
  }
});

---

Combining Voice and Image

You can use both features together:

// Image search + voice refinement
widget.on('image:result', ({ products }) => {
  // Show image results
  displayResults(products);

  // Suggest voice refinement
  showPrompt('Say "cheaper" or "Nike only" to refine');
});

widget.on('voice:end', ({ transcript }) => {
  // Refine image results with voice
  const currentFilters = widget.currentFilters;
  widget.search(transcript, { filters: currentFilters });
});

Accessibility

Both features are designed with accessibility in mind:

Voice

• ARIA announcement when microphone is active
• Visual + audio feedback
• Configurable timeout (10s default)

Image

• Descriptive button labels
• Accessible progress feedback
• Alt text for results

// Icons include aria-labels
<button aria-label="Search by voice">
<button aria-label="Search by image">

Disable Features

If you don't need these features:

NeuroonWidget.init({
  features: {
    voiceSearch: false,  // Hides microphone icon
    imageSearch: false,  // Hides camera icon
  },
});

Next steps

• Product Comparison - Compare results
• Accessibility - WCAG compliance
• JavaScript API - Programmatic control