Image Importer And Queue

This document describes how the ADP Car Market Hub plugin downloads, deduplicates and attaches vehicle images, and how the asynchronous image queue moves long-running image work out of the synchronous import path.

When to use this document

Read this document if you need to:

• Understand which images end up in the WordPress Media Library and which are reused or skipped.
• Diagnose missing, stale or duplicate vehicle images.
• Plan WebP conversion or storage capacity for a large catalogue.
• Tune cron-mode image processing for slow or restricted hosting.

For the surrounding import flow, see Import Engine. For the cron-side scheduling, see Cron Events And Scheduler.

Overview

Two classes share responsibility for image handling:

• AS24CI\Image_Importer — downloads a list of image URLs, optionally converts them to WebP, attaches them to a vehicle post via media_handle_sideload(), and records the source URL in attachment postmeta for deduplication.
• AS24CI\Scheduler — owns the asynchronous image queue. The worker hook as24ci_image_queue_process processes pending items in batches with its own transient lock and time budget.

The importer (Import Engine) decides whether to import images synchronously or to defer most of them to the queue, based on the cron mode of the current run.

Source of truth for images

For any imported attachment, the canonical "where did this come from?" marker is the _as24ci_source_url postmeta on the attachment post. The plugin uses this marker to:

• Deduplicate downloads — if an attachment with the same source URL already exists, the plugin reuses it rather than downloading again.
• Rebuild the vehicle's _as24ci_image_ids list authoritatively after the queue worker finishes, by querying child attachments with this meta key.

Two postmeta keys on the vehicle post track imported images:

• _as24ci_image_ids — array of WordPress attachment IDs that belong to the importer's gallery.
• _as24ci_images_hash — MD5 hash of the source image list. May hold the sentinel 'pending_queue' while items are still in the queue. The next import treats 'pending_queue' as a mismatch so the image list is always re-evaluated until it is fully imported.

Manually added gallery attachments are tracked separately in _as24ci_manual_image_ids and are never deleted by the plugin.

Synchronous image import

Image_Importer::import_images( int $post_id, array $images, string $listing_id ) performs the synchronous work:

1. Lazily includes WordPress media admin files (wp-admin/includes/media.php, file.php, image.php).
2. Defensively re-applies the as24ci_max_images cap.
3. Pre-fetches the post's known source URLs once via prefetch_known_urls_for_post() to avoid N per-image database queries.
4. For each image (string URL or array with url / src / href / original / imageUrl keys): - Reuses an existing attachment if the source URL is already known (in-memory map first, then a _as24ci_source_url postmeta lookup). - Otherwise calls download_url( $url, 30 ) (30-second timeout), optionally converts to WebP, and calls media_handle_sideload() to create the attachment. - Writes _as24ci_source_url on the new attachment and adds it to the in-memory deduplication map. - Logs failures (download errors, sideload errors) and moves on to the next image.
5. Merges the new attachment IDs into _as24ci_image_ids (preserving previously imported items) and sets the post thumbnail to the first attachment if the post does not already have one.

Existing IDs are merged rather than overwritten so that queue-worker calls accumulate attachments for the same post over multiple runs.

WebP conversion

When as24ci_convert_to_webp is 1, the importer attempts to convert each downloaded image to WebP before sideloading:

• Quality is read from as24ci_webp_quality (default 80 if unset) and clamped to the range 1–100. The as24ci_webp_quality filter can override the value.
• GD's imagewebp() is tried first; Imagick is used as a fallback.
• If neither extension is available, conversion is skipped and the original image is sideloaded. A log line is emitted.
• If conversion produces an empty file or fails, the temp file is cleaned up and the original image is used.

The converted file inherits the original filename with the extension replaced by .webp.

Cron-mode (queued) import

When the scheduler runs an import (cron or REST), it sets Importer::set_cron_image_queue( true ) if as24ci_cron_image_queue is 1 (the default). For each vehicle with more than one image, the importer:

1. Downloads only the first image immediately via the synchronous path so the vehicle has a thumbnail.
2. Pre-checks whether every remaining image is already attached (via the deduplication map). If yes, no queue entry is created and the final _as24ci_images_hash is written immediately.
3. Otherwise calls enqueue_remaining_images() to append items of the form { post_id, listing_id, url, images_hash } to the as24ci_image_queue option. The vehicle's _as24ci_images_hash is set to the sentinel 'pending_queue' so future imports keep re-evaluating until the queue completes.

The queue is hard-capped at Scheduler::IMAGE_QUEUE_SIZE_LIMIT items. New entries are dropped when the cap is reached and a log line is emitted.

Queue worker

AS24CI\Scheduler::run_image_queue() is bound to the as24ci_image_queue_process cron hook and runs the actual deferred downloads:

1. Calls set_time_limit(300) (subject to host restrictions).
2. Acquires the as24ci_image_queue_running transient lock with the Scheduler::IMAGE_QUEUE_LOCK_TTL TTL. A stale lock (older than the TTL) is automatically cleared.
3. Loads the queue from the as24ci_image_queue option.
4. Processes items in batches of Scheduler::IMAGE_QUEUE_BATCH_SIZE while the queue is non-empty and the time budget (IMAGE_QUEUE_LOCK_TTL − 60s) is not exhausted.
5. For each item, calls Image_Importer::import_images() for the single URL. Errors are caught and logged; the item is dropped to avoid a poison-pill loop.
6. Persists the remaining queue and writes per-run stats to as24ci_image_queue_last_run ({ processed, failed, remaining, timestamp }).
7. For every post fully drained from the queue this run, rebuilds _as24ci_image_ids authoritatively from the actual child attachments tagged with _as24ci_source_url and stores the final _as24ci_images_hash. This guarantees the meta is complete and consistent even if individual queue items failed.
8. Schedules the next run via maybe_schedule_image_queue_worker() if items remain.

The worker's images_hash value is validated to be a 32-character lowercase MD5 string before use. Corrupted entries are treated as absent so the post hash is not finalised from bad data.

Configuration reference

The exact values of IMAGE_QUEUE_LOCK_TTL, IMAGE_QUEUE_BATCH_SIZE and IMAGE_QUEUE_SIZE_LIMIT are defined as constants on AS24CI\Scheduler and may change between plugin versions. Verify them in the current source if you need to plan for a specific load.

Operational notes

• WP-Cron is the default trigger. On low-traffic sites consider a server-side cron / system cron pointing at wp-cron.php (with DISABLE_WP_CRON set to true) so the queue drains predictably.
• Image downloads are performed via download_url() which uses WordPress's HTTP API. Hosts that block outbound HTTP must whitelist the AutoScout24 image hosts.
• Image deduplication is based on the exact source URL. URLs that differ only in query string or signature parameters will be treated as distinct sources and downloaded separately.
• Manual gallery attachments tracked in _as24ci_manual_image_ids are never overwritten or removed by the importer or the queue worker.
• The queue worker rebuilds _as24ci_image_ids from the actual child attachments at the end of each fully-drained post; if you manipulate the meta directly between worker runs, your changes may be reverted.

Troubleshooting

• Vehicles appear without any images. Confirm as24ci_import_images is enabled. Check that as24ci_max_images is non-zero where intended. In cron-mode the primary image is imported synchronously; if no images appear at all, look for download errors in the plugin log.
• Only the first image is present after a cron import. This is expected during cron-mode imports until the as24ci_image_queue_process worker finishes. Check as24ci_image_queue_last_run for progress and the queue size.
• The queue grows but never drains. Verify that WP-Cron is running (or that a server cron triggers wp-cron.php). Check for a stuck as24ci_image_queue_running transient older than the worker's TTL. The worker auto-clears stale locks on the next run.
• WebP conversion does not happen. Make sure as24ci_convert_to_webp is 1 and that either GD with WebP support (imagewebp) or Imagick is installed. The plugin logs the reason when conversion is skipped.
• Duplicate-looking images appear in the gallery. The dedupe key is the exact source URL. CDNs that append unique query parameters per request will defeat dedupe. Verify the URLs in the API payload via verbose logging.
• Images disappeared after deleting a vehicle. The Vehicle_Deleter removes attachments tracked in _as24ci_image_ids. Manual gallery attachments (_as24ci_manual_image_ids) are preserved. Verify which list contained the affected attachment ID.

Related documents

• Import Engine
• Cron Events And Scheduler
• Data Model
• Database Schema
• Section overview