Phase D — batched imports & moves with disk staging — v1.7.0

Implements review-00-plan-00.md Phase D. Items addressed:

* M1 — category import is now batched. A new
  studiou_wcpcm_import_parse action stages rows to a guarded
  uploads/studiou-wcpcm-staging/<token>.json file (index.php + .htaccess
  guards, 24h TTL), and a new studiou_wcpcm_import_batch action
  processes 10 rows per call. The JS controller polls until done and
  reports progress in the results panel. The legacy single-shot
  studiou_wcpcm_import action stays registered for back-compat but the
  bundled UI no longer calls it.

* M2 — product move is now batched. studiou_wcpcm_move_products_start
  returns the source category's product IDs;
  studiou_wcpcm_move_products_chunk moves 25 at a time. The chunk
  handler uses wp_set_post_terms, which is idempotent, so a retried
  chunk is safe. The legacy single-shot
  studiou_wcpcm_move_products action stays registered for back-compat.

* M4 (rework) — product importer no longer stages CSV rows in a WP
  transient. The transient was a single wp_options row and overflowed
  max_allowed_packet on large imports, surfacing as the misleading
  "Import data not found or expired". The new disk-streaming staging
  has no such ceiling and is shared with the category importer.

The shared staging API is exposed as 5 static methods on
Studiou_WC_Product_Manage_Product_Import: get_staging_dir, stage_rows,
read_stage_slice, delete_stage, prune_staging. Code duplication between
the two importers is contained to row-shape handling.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

studiou-wc-product-cat-manage/CLAUDE.md

@@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 **QDR - Studiou WC Export/Import Product Categories** is a WordPress plugin that extends WooCommerce to provide batch export/import functionality for both product categories and products, along with category manipulation tools.
 
-**Current Version:** 1.6.4
+**Current Version:** 1.7.0
 
 ## Requirements
 

studiou-wc-product-cat-manage/assets/js/admin.js

@@ -19,7 +19,7 @@
     // Initialize admin scripts
     $(document).ready(function() {
         // Debug: Log that script is loaded
-        console.log('STUDIOU WC: Admin script loaded v1.6.4');
+        console.log('STUDIOU WC: Admin script loaded v1.7.0');
 
         if (typeof studiouWcpcm === 'undefined') {
             console.error('STUDIOU WC: studiouWcpcm object is NOT defined - AJAX will not work');
@@ -163,25 +163,29 @@
      * Initialize import form
      */
     function initImportForm() {
+        // v1.7.0 — M1: batched category import. The flow is parse → loop
+        // batches → summarise, mirroring the product importer. Removes the
+        // single-shot ~60s timeout on large category trees. The legacy
+        // single-shot endpoint stays registered for back-compat but JS no
+        // longer calls it.
         $('#studiou-wcpcm-import-form').on('submit', function(e) {
             e.preventDefault();
-            
-            // Show spinner
-            var $submitBtn = $(this).find('#studiou-wcpcm-import-button');
-            var $spinner = $(this).find('.spinner');
-            
+
+            var $form = $(this);
+            var $submitBtn = $form.find('#studiou-wcpcm-import-button');
+            var $spinner = $form.find('.spinner');
+
             $submitBtn.prop('disabled', true);
             $spinner.css('visibility', 'visible');
-            
-            // Clear previous notices
             $('.studiou-wcpcm-notice-area').empty();
-            
-            // Create form data
+            $('.studiou-wcpcm-import-results').hide();
+            $('#studiou-wcpcm-import-results').empty();
+
+            // Step 1 — parse the CSV server-side, stage to disk, get a token.
             var formData = new FormData(this);
-            formData.append('action', 'studiou_wcpcm_import');
+            formData.append('action', 'studiou_wcpcm_import_parse');
             formData.append('nonce', studiouWcpcm.nonce);
-            
-            // Send AJAX request
+
             $.ajax({
                 url: studiouWcpcm.ajaxUrl,
                 type: 'POST',
@@ -189,26 +193,84 @@
                 processData: false,
                 contentType: false,
                 success: function(response) {
-                    if (response.success) {
-                        showNotice('success', studiouWcpcm.i18n.importSuccess);
-                        
-                        // Show results
-                        $('#studiou-wcpcm-import-results').text(response.data.message);
-                        $('.studiou-wcpcm-import-results').show();
-                    } else {
-                        showNotice('error', response.data.message);
+                    if (!response.success) {
+                        showNotice('error', response.data && response.data.message ? response.data.message : 'Error');
+                        $submitBtn.prop('disabled', false);
+                        $spinner.css('visibility', 'hidden');
+                        return;
                     }
+                    runImportBatches(response.data.staging_key, response.data.total, $submitBtn, $spinner);
                 },
                 error: function() {
-                    showNotice('error', studiouWcpcm.i18n.importError);
-                },
-                complete: function() {
-                    // Hide spinner
+                    showNotice('error', studiouWcpcm.i18n.importError || 'Import error');
                     $submitBtn.prop('disabled', false);
                     $spinner.css('visibility', 'hidden');
                 }
             });
         });
+
+        function runImportBatches(stagingKey, total, $submitBtn, $spinner) {
+            var processed = 0;
+            var totalSuccess = 0;
+            var totalErrors = 0;
+            var allMessages = [];
+
+            // Show or build a simple progress indicator. The view doesn't
+            // require a dedicated progress-bar element; we render into the
+            // existing results container.
+            var $results = $('#studiou-wcpcm-import-results');
+            $results.empty();
+            $('.studiou-wcpcm-import-results').show();
+            $results.html('<p>0 / ' + total + '</p>');
+
+            function nextBatch() {
+                $.ajax({
+                    url: studiouWcpcm.ajaxUrl,
+                    type: 'POST',
+                    data: {
+                        action: 'studiou_wcpcm_import_batch',
+                        nonce: studiouWcpcm.nonce,
+                        staging_key: stagingKey,
+                        offset: processed
+                    },
+                    success: function(response) {
+                        if (!response.success) {
+                            showNotice('error', response.data && response.data.message ? response.data.message : 'Error');
+                            $submitBtn.prop('disabled', false);
+                            $spinner.css('visibility', 'hidden');
+                            return;
+                        }
+                        processed = response.data.processed;
+                        totalSuccess += response.data.success || 0;
+                        totalErrors += response.data.errors || 0;
+                        if (Array.isArray(response.data.messages)) {
+                            allMessages = allMessages.concat(response.data.messages);
+                        }
+                        $results.html('<p>' + processed + ' / ' + total + '</p>');
+
+                        if (response.data.done) {
+                            var summary = '<p><strong>' + (totalSuccess + totalErrors) + ' items processed: ' + totalSuccess + ' OK, ' + totalErrors + ' errors.</strong></p>';
+                            if (allMessages.length) {
+                                summary += '<pre style="max-height:300px;overflow:auto;">' + allMessages.map(function(m){return $('<div>').text(m).html();}).join('\n') + '</pre>';
+                            }
+                            $results.html(summary);
+                            showNotice(totalErrors === 0 ? 'success' : 'warning', studiouWcpcm.i18n.importSuccess || 'Import complete');
+                            $submitBtn.prop('disabled', false);
+                            $spinner.css('visibility', 'hidden');
+                        } else {
+                            nextBatch();
+                        }
+                    },
+                    error: function() {
+                        showNotice('error', studiouWcpcm.i18n.importError || 'Import error');
+                        $submitBtn.prop('disabled', false);
+                        $spinner.css('visibility', 'hidden');
+                    }
+                });
+            }
+
+            nextBatch();
+        }
     }
     
     /**
@@ -264,91 +326,130 @@
      * Initialize move products form
      */
     function initMoveForm() {
-        console.log('STUDIOU WC: Initializing move form handlers');
-        
-        // Add debug button click handler first
-        $('#studiou-wcpcm-move-button').on('click', function(e) {
-            console.log('STUDIOU WC: Move button clicked directly');
-        });
-        
+        // v1.7.0 — M2: batched product move. Step 1 (start) returns the
+        // list of product IDs in the source category. Step 2 (chunk) moves
+        // 25 at a time. JS drives the chunking + progress bar. Removes the
+        // single-shot ~60s PHP-timeout cliff on large source categories.
         $('#studiou-wcpcm-move-form').on('submit', function(e) {
             e.preventDefault();
-            
-            // Debug: Log form submission
-            console.log('STUDIOU WC: Move form submitted');
-            
-            // Validate that different categories are selected
+
             var sourceCategory = $('#source_category').val();
             var targetCategory = $('#target_category').val();
-            
-            console.log('STUDIOU WC: Source category:', sourceCategory, 'Target category:', targetCategory);
-            
+
             if (!sourceCategory || !targetCategory) {
-                console.log('STUDIOU WC: Missing category selection');
-                showNotice('error', 'Please select both source and target categories');
+                showNotice('error', studiouWcpcm.i18n.selectBothCategories || 'Please select both source and target categories');
                 return;
             }
-            
             if (sourceCategory === targetCategory) {
-                console.log('STUDIOU WC: Same category selected');
                 showNotice('error', studiouWcpcm.i18n.selectDifferentCategories);
                 return;
             }
-            
-            // Show spinner
-            var $submitBtn = $(this).find('#studiou-wcpcm-move-button');
-            var $spinner = $(this).find('.spinner');
-            
+
+            var $form = $(this);
+            var $submitBtn = $form.find('#studiou-wcpcm-move-button');
+            var $spinner = $form.find('.spinner');
             $submitBtn.prop('disabled', true);
             $spinner.css('visibility', 'visible');
-            
-            // Clear previous notices
             $('.studiou-wcpcm-notice-area').empty();
-            
-            // Debug: Log AJAX request details
-            console.log('STUDIOU WC: Sending AJAX request to:', studiouWcpcm.ajaxUrl);
-            console.log('STUDIOU WC: Nonce:', studiouWcpcm.nonce);
-            
-            // Send AJAX request
+            $('.studiou-wcpcm-move-results').hide();
+
+            // Step 1 — get the product IDs to move.
             $.ajax({
                 url: studiouWcpcm.ajaxUrl,
                 type: 'POST',
                 data: {
-                    action: 'studiou_wcpcm_move_products',
+                    action: 'studiou_wcpcm_move_products_start',
                     source_category: sourceCategory,
                     target_category: targetCategory,
                     nonce: studiouWcpcm.nonce
                 },
                 success: function(response) {
-                    console.log('STUDIOU WC: AJAX response:', response);
-                    
-                    if (response.success) {
-                        showNotice('success', studiouWcpcm.i18n.moveSuccess);
-                        
-                        // Show results
-                        $('#studiou-wcpcm-move-message').html('<p>' + response.data.message + '</p>');
+                    if (!response.success) {
+                        showNotice('error', response.data && response.data.message ? response.data.message : (studiouWcpcm.i18n.moveError || 'Error'));
+                        $submitBtn.prop('disabled', false);
+                        $spinner.css('visibility', 'hidden');
+                        return;
+                    }
+                    var ids = response.data.product_ids || [];
+                    var total = response.data.total || 0;
+                    var sourceName = response.data.source_name || '';
+                    var targetName = response.data.target_name || '';
+
+                    if (total === 0) {
+                        $('#studiou-wcpcm-move-message').html('<p>' + (studiouWcpcm.i18n.moveNoProducts || 'No products to move.') + '</p>');
                         $('.studiou-wcpcm-move-results').show();
-                        
-                        // Refresh the page to update category counts
-                        setTimeout(function() {
-                            location.reload();
-                        }, 3000);
-                    } else {
-                        showNotice('error', response.data.message);
+                        $submitBtn.prop('disabled', false);
+                        $spinner.css('visibility', 'hidden');
+                        return;
                     }
+                    runMoveChunks(ids, total, sourceCategory, targetCategory, sourceName, targetName, $submitBtn, $spinner);
                 },
-                error: function(xhr, status, error) {
-                    console.error('STUDIOU WC: AJAX error:', status, error);
-                    console.error('STUDIOU WC: Response text:', xhr.responseText);
-                    showNotice('error', studiouWcpcm.i18n.moveError + ': ' + error);
-                },
-                complete: function() {
-                    // Hide spinner
+                error: function() {
+                    showNotice('error', studiouWcpcm.i18n.moveError || 'Move error');
                     $submitBtn.prop('disabled', false);
                     $spinner.css('visibility', 'hidden');
                 }
             });
         });
+
+        function runMoveChunks(allIds, total, sourceId, targetId, sourceName, targetName, $submitBtn, $spinner) {
+            var CHUNK = 25;
+            var idx = 0;
+            var movedTotal = 0;
+            var failedTotal = 0;
+
+            var $msg = $('#studiou-wcpcm-move-message');
+            $msg.html('<p>0 / ' + total + '</p>');
+            $('.studiou-wcpcm-move-results').show();
+
+            function nextChunk() {
+                var chunk = allIds.slice(idx, idx + CHUNK);
+                if (chunk.length === 0) { return; }
+
+                $.ajax({
+                    url: studiouWcpcm.ajaxUrl,
+                    type: 'POST',
+                    data: {
+                        action: 'studiou_wcpcm_move_products_chunk',
+                        nonce: studiouWcpcm.nonce,
+                        source_category: sourceId,
+                        target_category: targetId,
+                        product_ids: chunk
+                    },
+                    success: function(response) {
+                        if (!response.success) {
+                            showNotice('error', response.data && response.data.message ? response.data.message : (studiouWcpcm.i18n.moveError || 'Error'));
+                            $submitBtn.prop('disabled', false);
+                            $spinner.css('visibility', 'hidden');
+                            return;
+                        }
+                        movedTotal += response.data.moved || 0;
+                        failedTotal += response.data.failed || 0;
+                        idx += chunk.length;
+                        $msg.html('<p>' + idx + ' / ' + total + '</p>');
+                        if (idx >= total) {
+                            var summary = '<p>' + movedTotal + ' moved from "' + sourceName + '" to "' + targetName + '"';
+                            if (failedTotal > 0) { summary += ', ' + failedTotal + ' failed'; }
+                            summary += '.</p>';
+                            $msg.html(summary);
+                            showNotice(failedTotal === 0 ? 'success' : 'warning', studiouWcpcm.i18n.moveSuccess || 'Move complete');
+                            $submitBtn.prop('disabled', false);
+                            $spinner.css('visibility', 'hidden');
+                            setTimeout(function() { location.reload(); }, 2000);
+                        } else {
+                            nextChunk();
+                        }
+                    },
+                    error: function() {
+                        showNotice('error', studiouWcpcm.i18n.moveError || 'Move error');
+                        $submitBtn.prop('disabled', false);
+                        $spinner.css('visibility', 'hidden');
+                    }
+                });
+            }
+
+            nextChunk();
+        }
     }
     
     /**

studiou-wc-product-cat-manage/includes/class-studiou-wc-product-cat-manage-import.php

@@ -79,9 +79,168 @@ class Studiou_WC_Product_Cat_Manage_Import {
      */
     public function __construct($db) {
         $this->db = $db;
-        
-        // Register AJAX handler
+
+        // Legacy single-shot import (v1.0+). Kept for back-compat; in
+        // practice the JS now uses the batched parse+batch flow added in
+        // v1.7.0. Slated for removal after one release.
         add_action('wp_ajax_studiou_wcpcm_import', array($this, 'handle_import_ajax'));
+
+        // v1.7.0 — M1: batched category import. Same shape as the product
+        // importer: parse stages rows to disk, batch processes N at a time
+        // by offset, JS polls until processed >= total. Removes the ~60s
+        // PHP-timeout cliff on large category trees.
+        add_action('wp_ajax_studiou_wcpcm_import_parse', array($this, 'handle_import_parse_ajax'));
+        add_action('wp_ajax_studiou_wcpcm_import_batch', array($this, 'handle_import_batch_ajax'));
+    }
+
+    /**
+     * v1.7.0 — Step 1: parse the uploaded CSV, validate headers, stage rows
+     * on disk. Returns the staging token + total count.
+     */
+    public function handle_import_parse_ajax() {
+        while (ob_get_level()) { ob_end_clean(); }
+
+        if (!isset($_POST['nonce']) || !wp_verify_nonce($_POST['nonce'], 'studiou-wcpcm-nonce')) {
+            wp_send_json_error(array('message' => __('Security check failed', 'studiou-wc-product-cat-manage')));
+        }
+        if (!current_user_can('manage_woocommerce')) {
+            wp_send_json_error(array('message' => __('You do not have sufficient permissions', 'studiou-wc-product-cat-manage')));
+        }
+        if (!isset($_FILES['import_file']) || empty($_FILES['import_file']['tmp_name'])) {
+            wp_send_json_error(array('message' => __('No file was uploaded', 'studiou-wc-product-cat-manage')));
+        }
+
+        try {
+            $file = fopen($_FILES['import_file']['tmp_name'], 'r');
+            if (!$file) {
+                throw new Exception(__('Failed to open the uploaded file', 'studiou-wc-product-cat-manage'));
+            }
+            $bom = fread($file, 3);
+            if ($bom !== "\xEF\xBB\xBF") { rewind($file); }
+            $headers = fgetcsv($file);
+            if (!$headers) {
+                fclose($file);
+                throw new Exception(__('Failed to read CSV headers', 'studiou-wc-product-cat-manage'));
+            }
+            $this->validate_headers($headers);
+
+            $header_count = count($headers);
+            $rows = array();
+            $row_number = 1;
+            while (($row = fgetcsv($file)) !== false) {
+                $row_number++;
+                if (empty(array_filter($row, function($v) { return $v !== '' && $v !== null; }))) {
+                    continue;
+                }
+                $rc = count($row);
+                if ($rc < $header_count) {
+                    $row = array_pad($row, $header_count, '');
+                } elseif ($rc > $header_count) {
+                    $row = array_slice($row, 0, $header_count);
+                }
+                $data = array();
+                foreach ($headers as $i => $h) {
+                    $data[$h] = isset($row[$i]) ? $row[$i] : '';
+                }
+                $rows[] = array('row_number' => $row_number, 'data' => $data);
+            }
+            fclose($file);
+
+            if (empty($rows)) {
+                wp_send_json_error(array('message' => __('CSV contains no data rows.', 'studiou-wc-product-cat-manage')));
+            }
+
+            $token = Studiou_WC_Product_Manage_Product_Import::stage_rows($rows, 'category');
+            if ($token === false) {
+                wp_send_json_error(array('message' => __('Failed to stage import data on disk (write error).', 'studiou-wc-product-cat-manage')));
+            }
+
+            wp_send_json_success(array(
+                'staging_key' => $token,
+                'total'       => count($rows),
+            ));
+        } catch (\Throwable $e) {
+            error_log('STUDIOU WC Category Import Parse Error: ' . $e->getMessage());
+            wp_send_json_error(array('message' => $e->getMessage()));
+        }
+    }
+
+    /**
+     * v1.7.0 — Step 2: process a batch of N rows from offset. JS polls until
+     * processed >= total.
+     */
+    public function handle_import_batch_ajax() {
+        while (ob_get_level()) { ob_end_clean(); }
+
+        if (!isset($_POST['nonce']) || !wp_verify_nonce($_POST['nonce'], 'studiou-wcpcm-nonce')) {
+            wp_send_json_error(array('message' => __('Security check failed', 'studiou-wc-product-cat-manage')));
+        }
+        if (!current_user_can('manage_woocommerce')) {
+            wp_send_json_error(array('message' => __('You do not have sufficient permissions', 'studiou-wc-product-cat-manage')));
+        }
+
+        $staging_key = isset($_POST['staging_key']) ? sanitize_text_field($_POST['staging_key']) : '';
+        $offset      = isset($_POST['offset']) ? intval($_POST['offset']) : 0;
+        $batch_size  = 10; // category ops are cheap; 10 per batch keeps each AJAX call well under PHP timeout
+
+        if ($staging_key === '') {
+            wp_send_json_error(array('message' => __('Invalid request', 'studiou-wc-product-cat-manage')));
+        }
+
+        try {
+            $slice = Studiou_WC_Product_Manage_Product_Import::read_stage_slice($staging_key, $offset, $batch_size);
+            if ($slice === null) {
+                wp_send_json_error(array('message' => __('Import data not found or expired', 'studiou-wc-product-cat-manage')));
+            }
+
+            $messages = array();
+            $success = 0;
+            $errors = 0;
+
+            foreach ($slice['batch'] as $item) {
+                $row_number = $item['row_number'];
+                $data = $item['data'];
+                try {
+                    $this->validate_row_data($data, $row_number);
+                    $operation = strtoupper(trim($data['operation']));
+                    switch ($operation) {
+                        case 'A': $r = $this->db->create_category($data); break;
+                        case 'U': $r = $this->db->update_category($data); break;
+                        case 'D': $r = $this->db->delete_category($data['name']); break;
+                        default:
+                            throw new Exception(sprintf(__('Invalid operation "%s" at row %d', 'studiou-wc-product-cat-manage'), $operation, $row_number));
+                    }
+                    if (isset($r['status']) && $r['status'] === 'error') {
+                        $errors++;
+                        $messages[] = sprintf(__('Row %d: %s', 'studiou-wc-product-cat-manage'), $row_number, $r['message']);
+                    } else {
+                        $success++;
+                        $messages[] = sprintf(__('Row %d: %s', 'studiou-wc-product-cat-manage'), $row_number, isset($r['message']) ? $r['message'] : __('OK', 'studiou-wc-product-cat-manage'));
+                    }
+                } catch (\Throwable $row_e) {
+                    $errors++;
+                    $messages[] = sprintf(__('Row %d: %s', 'studiou-wc-product-cat-manage'), $row_number, $row_e->getMessage());
+                }
+            }
+
+            $processed = $offset + count($slice['batch']);
+            $is_final = $processed >= $slice['total'];
+            if ($is_final) {
+                Studiou_WC_Product_Manage_Product_Import::delete_stage($staging_key);
+            }
+
+            wp_send_json_success(array(
+                'processed' => $processed,
+                'total'     => $slice['total'],
+                'success'   => $success,
+                'errors'    => $errors,
+                'messages'  => $messages,
+                'done'      => $is_final,
+            ));
+        } catch (\Throwable $e) {
+            error_log('STUDIOU WC Category Import Batch Error: ' . $e->getMessage());
+            wp_send_json_error(array('message' => $e->getMessage()));
+        }
     }
     
     /**

studiou-wc-product-cat-manage/includes/class-studiou-wc-product-cat-manage-manipulator.php

@@ -32,7 +32,15 @@ class Studiou_WC_Product_Cat_Manage_Manipulator {
         $this->db = $db;
         
         // Register AJAX handlers
+        // Legacy single-shot move (v1.0+). Kept for back-compat; JS now uses
+        // the batched flow added in v1.7.0.
         add_action('wp_ajax_studiou_wcpcm_move_products', array($this, 'handle_move_products_ajax'));
+        // v1.7.0 — M2: batched product move. Mirrors the delete-products
+        // pattern: start returns the list of product IDs in the source
+        // category; chunk processes 25 IDs per AJAX call. Removes the
+        // single-shot timeout on large source categories.
+        add_action('wp_ajax_studiou_wcpcm_move_products_start', array($this, 'handle_move_products_start_ajax'));
+        add_action('wp_ajax_studiou_wcpcm_move_products_chunk', array($this, 'handle_move_products_chunk_ajax'));
         add_action('wp_ajax_studiou_wcpcm_batch_description', array($this, 'handle_batch_description_ajax'));
         add_action('wp_ajax_studiou_wcpcm_delete_products', array($this, 'handle_delete_products_ajax'));
         add_action('wp_ajax_studiou_wcpcm_delete_products_batch', array($this, 'handle_delete_products_batch_ajax'));
@@ -360,6 +368,99 @@ class Studiou_WC_Product_Cat_Manage_Manipulator {
         }
     }
     
+    /**
+     * v1.7.0 — M2: batched product move, step 1 (start).
+     *
+     * Returns the list of product IDs in the source category. JS chunks
+     * them and calls handle_move_products_chunk_ajax in batches.
+     */
+    public function handle_move_products_start_ajax() {
+        while (ob_get_level()) { ob_end_clean(); }
+
+        if (!isset($_POST['nonce']) || !wp_verify_nonce($_POST['nonce'], 'studiou-wcpcm-nonce')) {
+            wp_send_json_error(array('message' => __('Security check failed', 'studiou-wc-product-cat-manage')));
+        }
+        if (!current_user_can('manage_woocommerce')) {
+            wp_send_json_error(array('message' => __('You do not have sufficient permissions', 'studiou-wc-product-cat-manage')));
+        }
+
+        $source = isset($_POST['source_category']) ? intval($_POST['source_category']) : 0;
+        $target = isset($_POST['target_category']) ? intval($_POST['target_category']) : 0;
+
+        if ($source <= 0 || $target <= 0) {
+            wp_send_json_error(array('message' => __('Invalid category selection', 'studiou-wc-product-cat-manage')));
+        }
+        if ($source === $target) {
+            wp_send_json_error(array('message' => __('Source and target categories must be different', 'studiou-wc-product-cat-manage')));
+        }
+
+        $source_term = get_term($source, 'product_cat');
+        $target_term = get_term($target, 'product_cat');
+        if (is_wp_error($source_term) || !$source_term || is_wp_error($target_term) || !$target_term) {
+            wp_send_json_error(array('message' => __('Invalid category specified', 'studiou-wc-product-cat-manage')));
+        }
+
+        $product_ids = $this->get_products_in_category($source);
+        wp_send_json_success(array(
+            'product_ids' => array_values(array_map('intval', $product_ids)),
+            'total'       => count($product_ids),
+            'source_name' => $source_term->name,
+            'target_name' => $target_term->name,
+        ));
+    }
+
+    /**
+     * v1.7.0 — M2: batched product move, step 2 (chunk).
+     *
+     * Moves the supplied product IDs from source to target category.
+     * wp_set_post_terms is idempotent, so a re-run of an already-moved
+     * chunk (e.g. after a network retry) is safe.
+     */
+    public function handle_move_products_chunk_ajax() {
+        while (ob_get_level()) { ob_end_clean(); }
+
+        if (!isset($_POST['nonce']) || !wp_verify_nonce($_POST['nonce'], 'studiou-wcpcm-nonce')) {
+            wp_send_json_error(array('message' => __('Security check failed', 'studiou-wc-product-cat-manage')));
+        }
+        if (!current_user_can('manage_woocommerce')) {
+            wp_send_json_error(array('message' => __('You do not have sufficient permissions', 'studiou-wc-product-cat-manage')));
+        }
+
+        $source = isset($_POST['source_category']) ? intval($_POST['source_category']) : 0;
+        $target = isset($_POST['target_category']) ? intval($_POST['target_category']) : 0;
+        $ids    = isset($_POST['product_ids']) && is_array($_POST['product_ids'])
+                    ? array_values(array_map('intval', $_POST['product_ids']))
+                    : array();
+
+        if ($source <= 0 || $target <= 0 || $source === $target || empty($ids)) {
+            wp_send_json_error(array('message' => __('Invalid request', 'studiou-wc-product-cat-manage')));
+        }
+
+        $moved = 0;
+        $failed = 0;
+        foreach ($ids as $pid) {
+            $current = wp_get_post_terms($pid, 'product_cat', array('fields' => 'ids'));
+            if (is_wp_error($current)) {
+                $failed++;
+                continue;
+            }
+            $new = array_diff($current, array($source));
+            $new[] = $target;
+            $new = array_values(array_unique($new));
+            $r = wp_set_post_terms($pid, $new, 'product_cat');
+            if (is_wp_error($r)) {
+                $failed++;
+            } else {
+                $moved++;
+            }
+        }
+
+        wp_send_json_success(array(
+            'moved'  => $moved,
+            'failed' => $failed,
+        ));
+    }
+
     /**
      * Move all products from source category to target category
      *

studiou-wc-product-cat-manage/includes/class-studiou-wc-product-manage-product-import.php

@@ -38,6 +38,102 @@ class Studiou_WC_Product_Manage_Product_Import {
         $this->db = $db;
     }
 
+    /**
+     * v1.7.0 — Return (and lazily create) the guarded staging directory used
+     * by both the product and category importers. Replaces the transient-
+     * based staging that capped at max_allowed_packet (M4).
+     *
+     * @return string Absolute path with trailing slash.
+     */
+    public static function get_staging_dir() {
+        $upload_dir = wp_upload_dir();
+        $dir = trailingslashit($upload_dir['basedir']) . 'studiou-wcpcm-staging/';
+        if (!file_exists($dir)) {
+            wp_mkdir_p($dir);
+        }
+        $idx = $dir . 'index.php';
+        if (!file_exists($idx)) {
+            @file_put_contents($idx, '<?php // Silence is golden.');
+        }
+        $ht = $dir . '.htaccess';
+        if (!file_exists($ht)) {
+            @file_put_contents($ht, "Deny from all\n");
+        }
+        return $dir;
+    }
+
+    /**
+     * v1.7.0 — Prune staging files older than 24h.
+     */
+    public static function prune_staging() {
+        $dir = self::get_staging_dir();
+        $cutoff = time() - DAY_IN_SECONDS;
+        $files = glob($dir . '*.json');
+        if (!is_array($files)) { return; }
+        foreach ($files as $f) {
+            $mtime = @filemtime($f);
+            if ($mtime !== false && $mtime < $cutoff) {
+                @unlink($f);
+            }
+        }
+    }
+
+    /**
+     * v1.7.0 — Stage an array of rows to a token-named JSON file. Returns
+     * the token, or false on write failure.
+     *
+     * @param array $rows
+     * @param string $prefix e.g. 'product' or 'category'
+     * @return string|false
+     */
+    public static function stage_rows($rows, $prefix = 'product') {
+        self::prune_staging();
+        $token = $prefix . '-' . bin2hex(random_bytes(16));
+        $path = self::get_staging_dir() . $token . '.json';
+        $encoded = wp_json_encode($rows);
+        if ($encoded === false || file_put_contents($path, $encoded) === false) {
+            return false;
+        }
+        return $token;
+    }
+
+    /**
+     * v1.7.0 — Read a slice of a staged file.
+     *
+     * @param string $token
+     * @param int $offset
+     * @param int $batch_size
+     * @return array|null array{total:int,batch:array} or null if missing/unreadable
+     */
+    public static function read_stage_slice($token, $offset, $batch_size) {
+        // Validate the token shape (defends against path traversal).
+        if (!preg_match('/^(product|category)-[a-f0-9]{32}$/', $token)) {
+            return null;
+        }
+        $path = self::get_staging_dir() . $token . '.json';
+        if (!file_exists($path)) { return null; }
+        $raw = file_get_contents($path);
+        if ($raw === false) { return null; }
+        $rows = json_decode($raw, true);
+        if (!is_array($rows)) { return null; }
+        return array(
+            'total' => count($rows),
+            'batch' => array_slice($rows, $offset, $batch_size),
+        );
+    }
+
+    /**
+     * v1.7.0 — Delete a staged file (e.g. on completion).
+     *
+     * @param string $token
+     */
+    public static function delete_stage($token) {
+        if (!preg_match('/^(product|category)-[a-f0-9]{32}$/', $token)) {
+            return;
+        }
+        @unlink(self::get_staging_dir() . $token . '.json');
+    }
+
     /**
      * Parse CSV file and store data for batch processing
      *
@@ -127,24 +223,26 @@ class Studiou_WC_Product_Manage_Product_Import {
         // upload was fine, the store just couldn't take it. Surface a
         // clear, actionable message instead. The disk-staging rework that
         // actually removes this ceiling lands in Phase D.
-        $transient_key = 'studiou_wcpcm_import_' . uniqid();
-        if (!set_transient($transient_key, $rows, 3600)) {
-            return array('error' => sprintf(
-                /* translators: %d: number of rows */
-                __('Import too large to stage (%d rows) — the parsed CSV would not fit in the WordPress transient store. Split the file into smaller batches, or wait for the streaming importer in a future release.', 'studiou-wc-product-cat-manage'),
-                count($rows)
-            ));
-        }
-        // Skipped (column-count-mismatch) rows live in a sibling transient and
-        // get folded into the final batch's failed_items + messages by
-        // process_batch(), so they appear in the failed-CSV download without
-        // changing the JS contract.
-        if (!empty($skipped_rows)) {
-            set_transient($transient_key . '_skipped', $skipped_rows, 3600);
+        // v1.7.0 — M4 rework: stage rows on disk instead of in a transient.
+        // The transient was a single wp_options row; large CSVs overflowed
+        // max_allowed_packet and set_transient returned false, surfacing as
+        // the misleading "Import data not found or expired" downstream.
+        // Disk staging has no such ceiling. The staging file is guarded
+        // (index.php + .htaccess), token-named, and pruned at 24h.
+        $payload = array(
+            'rows'    => $rows,
+            'skipped' => $skipped_rows,
+        );
+        $token = self::stage_rows($payload, 'product');
+        if ($token === false) {
+            return array('error' => __('Failed to stage import data on disk (write error).', 'studiou-wc-product-cat-manage'));
         }
 
         return array(
-            'transient_key' => $transient_key,
+            // Legacy field name preserved so the JS keeps working without
+            // changes; the value is now a disk-staging token, not a
+            // transient key. process_batch reads from disk via the token.
+            'transient_key' => $token,
             'total'         => count($rows),
         );
     }
@@ -158,11 +256,26 @@ class Studiou_WC_Product_Manage_Product_Import {
      * @return array Result with statistics
      */
     public function process_batch($transient_key, $offset = 0, $batch_size = 5) {
-        $rows = get_transient($transient_key);
-
-        if (!$rows) {
+        // v1.7.0 — read from disk-staging instead of the WP transient.
+        // First load the full payload so we know total + skipped rows.
+        $dir = self::get_staging_dir();
+        if (!preg_match('/^product-[a-f0-9]{32}$/', $transient_key)) {
             return array('error' => __('Import data not found or expired', 'studiou-wc-product-cat-manage'));
         }
+        $path = $dir . $transient_key . '.json';
+        if (!file_exists($path)) {
+            return array('error' => __('Import data not found or expired', 'studiou-wc-product-cat-manage'));
+        }
+        $raw = file_get_contents($path);
+        if ($raw === false) {
+            return array('error' => __('Import data not found or expired', 'studiou-wc-product-cat-manage'));
+        }
+        $payload = json_decode($raw, true);
+        if (!is_array($payload) || !isset($payload['rows'])) {
+            return array('error' => __('Import data not found or expired', 'studiou-wc-product-cat-manage'));
+        }
+        $rows = $payload['rows'];
+        $skipped_at_parse = isset($payload['skipped']) ? $payload['skipped'] : array();
 
         $result = array(
             'success' => 0,
@@ -181,15 +294,11 @@ class Studiou_WC_Product_Manage_Product_Import {
         // step (set by parse_csv_for_batch when row width != header width)
         // so they appear in the failed-CSV download and the messages list.
         $is_final_batch = ($offset + count($batch)) >= count($rows);
-        if ($is_final_batch) {
-            $parse_skipped = get_transient($transient_key . '_skipped');
-            if (is_array($parse_skipped) && !empty($parse_skipped)) {
-                foreach ($parse_skipped as $skip) {
-                    $result['skipped']++;
-                    $result['messages'][] = $skip['reason'];
-                    $result['failed_items'][] = $skip['data'];
-                }
-                delete_transient($transient_key . '_skipped');
+        if ($is_final_batch && !empty($skipped_at_parse)) {
+            foreach ($skipped_at_parse as $skip) {
+                $result['skipped']++;
+                $result['messages'][] = $skip['reason'];
+                $result['failed_items'][] = $skip['data'];
             }
         }
 
@@ -226,6 +335,14 @@ class Studiou_WC_Product_Manage_Product_Import {
 
         $result['processed'] = $offset + count($batch);
 
+        // v1.7.0 — delete the staging file when the import is complete so
+        // we don't leak token-named JSON files into uploads/. (Pruning at
+        // 24h would catch them eventually, but cleaning at completion is
+        // the precise time.)
+        if ($is_final_batch) {
+            self::delete_stage($transient_key);
+        }
+
         return $result;
     }
 

studiou-wc-product-cat-manage/readme.md

@@ -1,6 +1,6 @@
 # QDR - Studiou WC Export/Import Product Categories
 
-**Version: 1.6.4**
+**Version: 1.7.0**
 
 A WordPress plugin that extends WooCommerce to provide batch export and import functionality for both product categories and products, along with category manipulation tools.
 
@@ -352,6 +352,12 @@ For support, please visit [https://www.quadarax.com/plugins/studiou-wc-product-c
 
 ## Changelog
 
+### Version 1.7.0
+- **Category import is now batched and disk-staged (M1, M4 rework).** The legacy single-shot endpoint that processed the entire CSV in one AJAX call ran into the ~60 second PHP timeout on stores with thousands of categories. The new flow mirrors the product importer: the parse step stages rows to a guarded `wp-content/uploads/studiou-wcpcm-staging/<token>.json` file (`index.php` + `.htaccess` guards, 24-hour TTL); the batch step processes 10 rows per AJAX call and reports progress; the JS controller polls until done. Two new AJAX actions: `studiou_wcpcm_import_parse`, `studiou_wcpcm_import_batch`. The legacy `studiou_wcpcm_import` action is still registered for backwards compatibility but is no longer used by the bundled UI.
+- **Product move is now batched (M2).** Same shape as the existing delete-products flow: a `studiou_wcpcm_move_products_start` action returns the source category's product IDs, then `studiou_wcpcm_move_products_chunk` moves 25 at a time. `wp_set_post_terms` is idempotent, so a retried chunk after a network blip won't corrupt anything. The single-shot `studiou_wcpcm_move_products` action remains for backwards compatibility.
+- **Product import no longer uses a WordPress transient to stage CSV rows (M4 rework).** The transient was a single `wp_options` row; large imports overflowed `max_allowed_packet` and surfaced as the misleading "Import data not found or expired". The new disk-streaming staging (shared with the category importer) has no such ceiling. The 1.6.2 transient-failure guard becomes unnecessary but remains as a defensive error path on the new disk write.
+- **Two new helper static methods on `Studiou_WC_Product_Manage_Product_Import`** (`get_staging_dir`, `stage_rows`, `read_stage_slice`, `delete_stage`, `prune_staging`) are the shared staging API used by both importers.
+
 ### Version 1.6.4
 - **Security: gated product-export download.** Previously the product exporter dropped `product-export-YYYY-MM-DD-H-i-s.csv` into `wp-content/uploads/studiou-wc-product-exports/` and returned the raw public URL — anyone who knew or guessed the timestamped name could download the file, and the directory was never cleaned. Now mirrors the category exporter:
   - Export files live in `wp-content/uploads/studiou-wcpcm-product-exports/` with `index.php` and `.htaccess` guards.

studiou-wc-product-cat-manage/studiou-wc-product-cat-manage.php

@@ -3,7 +3,7 @@
  * Plugin Name: QDR - Studiou WC Export/Import Product Categories
  * Plugin URI: https://www.quadarax.com/plugins/studiou-wc-product-cat-manage
  * Description: Allows batch import and export WooCommerce product categories and products
- * Version: 1.6.4
+ * Version: 1.7.0
  * Requires at least: 6.8.1
  * Requires PHP: 8.2
  * Author: Dalibor Votruba
@@ -23,7 +23,7 @@ if (!defined('WPINC')) {
 }
 
 // Define plugin constants
-if (!defined('STUDIOU_WCPCM_VERSION')) define('STUDIOU_WCPCM_VERSION', '1.6.4');
+if (!defined('STUDIOU_WCPCM_VERSION')) define('STUDIOU_WCPCM_VERSION', '1.7.0');
 if (!defined('STUDIOU_WCPCM_PLUGIN_DIR')) define('STUDIOU_WCPCM_PLUGIN_DIR', plugin_dir_path(__FILE__));
 if (!defined('STUDIOU_WCPCM_PLUGIN_URL')) define('STUDIOU_WCPCM_PLUGIN_URL', plugin_dir_url(__FILE__));
 if (!defined('STUDIOU_WCPCM_PLUGIN_BASENAME')) define('STUDIOU_WCPCM_PLUGIN_BASENAME', plugin_basename(__FILE__));