@@ -0,0 +1,111 @@
|
|||||||
|
# VisionFlow Automator - Performance Optimizations
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
This document outlines the performance and memory optimizations implemented to prevent the application from being killed or stalled during scenario execution.
|
||||||
|
|
||||||
|
## Key Optimizations Implemented
|
||||||
|
|
||||||
|
### 1. Memory Management
|
||||||
|
- **Template Caching**: Added `TemplateCache` class to cache loaded CV2 templates instead of reloading from disk every loop iteration
|
||||||
|
- **Screenshot Caching**: Implemented screenshot caching with 50ms duration to reduce memory allocations
|
||||||
|
- **Resource Cleanup**: Added proper cleanup in `cleanup_resources()` method and `closeEvent()`
|
||||||
|
- **Garbage Collection**: Strategic `gc.collect()` calls to force memory cleanup at appropriate times
|
||||||
|
- **Pixmap Memory Management**: Optimized image preview handling to prevent large pixmap memory leaks
|
||||||
|
|
||||||
|
### 2. Performance Improvements
|
||||||
|
- **Optimized Loop Structure**: Redesigned automation loop to be more efficient
|
||||||
|
- **Step Cooldown System**: Prevents rapid re-execution of the same step (1-second cooldown)
|
||||||
|
- **Priority Execution**: Only executes one step per loop iteration to prevent blocking
|
||||||
|
- **Dynamic Sleep**: Adjusts sleep time based on loop performance (0.05-0.2s)
|
||||||
|
- **Performance Monitoring**: Added loop time tracking and warnings for slow performance
|
||||||
|
|
||||||
|
### 3. Threading Improvements
|
||||||
|
- **Proper Thread Management**: Added timeout for thread joining in `stop_automation()`
|
||||||
|
- **Background Processing**: Improved worker thread handling with better error recovery
|
||||||
|
- **Non-blocking Operations**: Reduced blocking operations in the main thread
|
||||||
|
|
||||||
|
### 4. UI Optimizations
|
||||||
|
- **Reduced Logging**: Changed default log level from DEBUG to INFO to reduce I/O overhead
|
||||||
|
- **Efficient Image Previews**: Optimized image scaling and memory usage in dialogs
|
||||||
|
- **State Update Throttling**: Reduced frequency of UI state updates
|
||||||
|
|
||||||
|
### 5. Resource Monitoring
|
||||||
|
- **Performance Timer**: Added 10-second monitoring timer to track application health
|
||||||
|
- **Memory Usage Tracking**: Optional psutil integration for detailed memory monitoring
|
||||||
|
- **Cache Size Monitoring**: Tracks template cache size and warns when it grows large
|
||||||
|
|
||||||
|
### 6. Error Handling
|
||||||
|
- **Graceful Degradation**: Better error handling in screenshot capture and template matching
|
||||||
|
- **Resource Recovery**: Automatic cleanup on errors to prevent resource leaks
|
||||||
|
- **Hotkey Error Handling**: Continues operation even if hotkey setup fails
|
||||||
|
|
||||||
|
## Configuration Changes
|
||||||
|
|
||||||
|
### PyAutoGUI Optimizations
|
||||||
|
- Set `pyautogui.PAUSE = 0.01` (reduced from default 0.1s)
|
||||||
|
- Set `pyautogui.MINIMUM_DURATION = 0` for faster actions
|
||||||
|
- Kept `pyautogui.FAILSAFE = False` for automation reliability
|
||||||
|
|
||||||
|
### Screenshot Optimization
|
||||||
|
- Added minimum size validation (10x10 pixels)
|
||||||
|
- Improved tkinter screenshot tool with better UX
|
||||||
|
- Memory-efficient PIL image handling with proper cleanup
|
||||||
|
|
||||||
|
### Template Matching
|
||||||
|
- LRU cache with automatic cleanup of old entries
|
||||||
|
- Maximum cache size of 50 templates
|
||||||
|
- Force reload option for template updates
|
||||||
|
|
||||||
|
## Performance Targets
|
||||||
|
|
||||||
|
### Memory Usage
|
||||||
|
- Template cache limited to 50 entries
|
||||||
|
- Screenshot cache duration: 50ms
|
||||||
|
- Automatic cleanup every 100 loop iterations
|
||||||
|
- Garbage collection after major operations
|
||||||
|
|
||||||
|
### Timing
|
||||||
|
- Target loop time: 50-200ms
|
||||||
|
- Step cooldown: 1 second
|
||||||
|
- Performance warning threshold: 1 second average loop time
|
||||||
|
- Monitoring interval: 10 seconds
|
||||||
|
|
||||||
|
### Thread Safety
|
||||||
|
- Worker thread timeout: 2 seconds
|
||||||
|
- Daemon threads for automatic cleanup
|
||||||
|
- Proper resource locking where needed
|
||||||
|
|
||||||
|
## Usage Recommendations
|
||||||
|
|
||||||
|
1. **Monitor Performance**: Check logs for performance warnings
|
||||||
|
2. **Resource Management**: Regularly restart long-running sessions
|
||||||
|
3. **Template Optimization**: Use appropriately sized template images
|
||||||
|
4. **Step Design**: Avoid too many simultaneous image detections
|
||||||
|
5. **System Resources**: Ensure adequate RAM for screenshot operations
|
||||||
|
|
||||||
|
## Future Improvements
|
||||||
|
|
||||||
|
1. **Multi-threading**: Consider separate threads for image processing
|
||||||
|
2. **Image Compression**: Compress cached templates to save memory
|
||||||
|
3. **Region Optimization**: Use smaller detection regions when possible
|
||||||
|
4. **GPU Acceleration**: Consider OpenCV GPU operations for template matching
|
||||||
|
5. **Background Processing**: Process non-critical operations in background
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### High Memory Usage
|
||||||
|
- Check template cache size in logs
|
||||||
|
- Reduce number of simultaneous steps
|
||||||
|
- Restart application periodically
|
||||||
|
|
||||||
|
### Slow Performance
|
||||||
|
- Check average loop times in logs
|
||||||
|
- Reduce image template sizes
|
||||||
|
- Simplify detection regions
|
||||||
|
- Consider fewer simultaneous detections
|
||||||
|
|
||||||
|
### Application Crashes
|
||||||
|
- Check log files for error patterns
|
||||||
|
- Monitor system memory usage
|
||||||
|
- Verify image file integrity
|
||||||
|
- Check for corrupted templates
|
||||||
@@ -0,0 +1,71 @@
|
|||||||
|
# VisionFlow Automator - Performance Optimization Summary
|
||||||
|
|
||||||
|
## Optimizations Applied
|
||||||
|
|
||||||
|
✅ **Memory Management**
|
||||||
|
- Added template caching system to prevent repeated file I/O
|
||||||
|
- Implemented screenshot caching with 50ms duration
|
||||||
|
- Added proper resource cleanup and garbage collection
|
||||||
|
- Optimized image preview handling to prevent memory leaks
|
||||||
|
|
||||||
|
✅ **Performance Improvements**
|
||||||
|
- Redesigned automation loop for better efficiency
|
||||||
|
- Added step cooldown system (1-second minimum between executions)
|
||||||
|
- Implemented dynamic sleep timing based on performance
|
||||||
|
- Added performance monitoring and warning system
|
||||||
|
|
||||||
|
✅ **Threading & Stability**
|
||||||
|
- Improved worker thread management with proper cleanup
|
||||||
|
- Added timeout handling for thread termination
|
||||||
|
- Better error handling to prevent application crashes
|
||||||
|
- Non-blocking operations in main UI thread
|
||||||
|
|
||||||
|
✅ **Resource Optimization**
|
||||||
|
- Reduced logging verbosity to decrease I/O overhead
|
||||||
|
- Optimized PyAutoGUI settings for faster execution
|
||||||
|
- Improved screenshot selection tool with better UX
|
||||||
|
- Added periodic cleanup every 100 loop iterations
|
||||||
|
|
||||||
|
## Key Features Added
|
||||||
|
|
||||||
|
### TemplateCache Class
|
||||||
|
- LRU cache for CV2 template images
|
||||||
|
- Automatic cleanup of old entries
|
||||||
|
- Maximum 50 templates cached
|
||||||
|
- Memory-efficient template loading
|
||||||
|
|
||||||
|
### Performance Monitoring
|
||||||
|
- Loop time tracking and warnings
|
||||||
|
- Memory usage monitoring
|
||||||
|
- 10-second monitoring intervals
|
||||||
|
- Automatic performance alerts
|
||||||
|
|
||||||
|
### Optimized Automation Loop
|
||||||
|
- Priority-based step execution
|
||||||
|
- Screenshot caching and reuse
|
||||||
|
- Improved error handling and recovery
|
||||||
|
- Dynamic timing adjustments
|
||||||
|
|
||||||
|
## Expected Results
|
||||||
|
|
||||||
|
- **Reduced Memory Usage**: Template caching and proper cleanup
|
||||||
|
- **Faster Performance**: Optimized loops and reduced I/O
|
||||||
|
- **Better Stability**: Improved error handling and resource management
|
||||||
|
- **No More Stalling**: Non-blocking operations and better threading
|
||||||
|
|
||||||
|
## Configuration Changes
|
||||||
|
|
||||||
|
```python
|
||||||
|
# PyAutoGUI optimizations
|
||||||
|
pyautogui.PAUSE = 0.01 # Faster actions
|
||||||
|
pyautogui.MINIMUM_DURATION = 0 # No minimum delays
|
||||||
|
|
||||||
|
# Logging optimization
|
||||||
|
logging.basicConfig(level=logging.INFO) # Reduced verbosity
|
||||||
|
|
||||||
|
# Template cache settings
|
||||||
|
max_cache_size = 50 # Maximum cached templates
|
||||||
|
cache_cleanup_threshold = 25% # When to clean old entries
|
||||||
|
```
|
||||||
|
|
||||||
|
The optimized application should now run more efficiently without getting killed or stalled during scenario execution.
|
||||||
@@ -0,0 +1,90 @@
|
|||||||
|
# Resource Usage Display - Implementation Summary
|
||||||
|
|
||||||
|
## ✅ **Features Successfully Added**
|
||||||
|
|
||||||
|
### 🎯 **Main UI Enhancements**
|
||||||
|
- **Resource Usage Group Box**: New section displaying real-time resource metrics
|
||||||
|
- **Toggle Button**: Hide/Show resources button to save screen space
|
||||||
|
- **Color-Coded Indicators**: Visual feedback for performance status
|
||||||
|
- **Interactive Help**: Click to get psutil installation instructions
|
||||||
|
|
||||||
|
### 📊 **Monitoring Metrics**
|
||||||
|
1. **Memory Usage**: Process memory in MB and percentage
|
||||||
|
2. **CPU Usage**: Process CPU utilization percentage
|
||||||
|
3. **System Memory**: Overall system memory usage and available space
|
||||||
|
4. **Cache Information**: Template cache size and cooldown entries
|
||||||
|
5. **Performance**: Average automation loop execution time
|
||||||
|
|
||||||
|
### 🔄 **Real-Time Updates**
|
||||||
|
- **2-Second Refresh**: Resource display updates every 2 seconds
|
||||||
|
- **Non-Blocking**: Updates don't interfere with automation performance
|
||||||
|
- **Automatic Monitoring**: Starts immediately when application launches
|
||||||
|
|
||||||
|
### 🎨 **Visual Design**
|
||||||
|
- **Color-Coded Status**:
|
||||||
|
- 🟢 Green: Normal/Good performance
|
||||||
|
- 🟡 Orange: Warning levels
|
||||||
|
- 🔴 Red: High usage/Performance issues
|
||||||
|
- ⚫ Gray: Unavailable metrics
|
||||||
|
- **Compact Layout**: Fits seamlessly into existing UI
|
||||||
|
- **Professional Styling**: Consistent with application theme
|
||||||
|
|
||||||
|
### 💡 **Smart Features**
|
||||||
|
- **Fallback Mode**: Works without psutil, shows basic cache information
|
||||||
|
- **Enhanced Mode**: Full system metrics when psutil is installed
|
||||||
|
- **Error Handling**: Graceful degradation if monitoring fails
|
||||||
|
- **Installation Guide**: Built-in help for psutil setup
|
||||||
|
|
||||||
|
## 🚀 **Benefits**
|
||||||
|
|
||||||
|
### For Performance Monitoring
|
||||||
|
- **Real-Time Visibility**: See exactly how much resources the app uses
|
||||||
|
- **Performance Bottlenecks**: Identify slow automation loops
|
||||||
|
- **Memory Leak Detection**: Monitor memory usage over time
|
||||||
|
- **System Impact**: Understand effect on overall system performance
|
||||||
|
|
||||||
|
### For Troubleshooting
|
||||||
|
- **Quick Diagnosis**: Color-coded warnings for immediate issue identification
|
||||||
|
- **Cache Monitoring**: Track template cache efficiency
|
||||||
|
- **Resource Optimization**: Data to optimize scenario performance
|
||||||
|
- **System Health**: Monitor system memory and CPU usage
|
||||||
|
|
||||||
|
## 📝 **Usage Instructions**
|
||||||
|
|
||||||
|
### Basic Usage
|
||||||
|
1. **View Resources**: Resource panel is visible by default at the bottom of the main window
|
||||||
|
2. **Toggle Display**: Click "Hide Resources" button to minimize screen usage
|
||||||
|
3. **Monitor Status**: Watch color changes for performance alerts
|
||||||
|
|
||||||
|
### Enhanced Monitoring
|
||||||
|
1. **Install psutil**: Run `pip install psutil` (already added to requirements.txt)
|
||||||
|
2. **Restart App**: Close and reopen VisionFlow Automator
|
||||||
|
3. **Full Metrics**: All detailed system information will be displayed
|
||||||
|
|
||||||
|
### Reading the Display
|
||||||
|
- **Memory**: Shows process memory usage (green < 10%, red > 10%)
|
||||||
|
- **CPU**: Shows process CPU usage (green < 50%, red > 50%)
|
||||||
|
- **System**: Shows system memory usage (green < 85%, red > 85%)
|
||||||
|
- **Cache**: Shows template cache size (blue normal, orange > 20 entries)
|
||||||
|
- **Performance**: Shows loop execution time (green < 500ms, red > 500ms)
|
||||||
|
|
||||||
|
## 🔧 **Technical Implementation**
|
||||||
|
|
||||||
|
### Code Changes
|
||||||
|
- Enhanced `get_memory_usage()` method with comprehensive metrics
|
||||||
|
- Added `_update_resource_display()` for UI updates
|
||||||
|
- Modified `_monitor_performance()` for real-time monitoring
|
||||||
|
- Added resource display widgets to main UI layout
|
||||||
|
- Implemented toggle functionality for space-saving
|
||||||
|
|
||||||
|
### Error Handling
|
||||||
|
- Graceful fallback when psutil unavailable
|
||||||
|
- Error display in resource panel for troubleshooting
|
||||||
|
- Maintains basic functionality even if monitoring fails
|
||||||
|
|
||||||
|
### Performance Impact
|
||||||
|
- Minimal overhead (2-second update cycle)
|
||||||
|
- Non-blocking UI updates
|
||||||
|
- Optimized memory usage for monitoring itself
|
||||||
|
|
||||||
|
The resource usage display provides valuable real-time insights into application performance and helps users optimize their automation scenarios for better efficiency and system resource management.
|
||||||
@@ -0,0 +1,143 @@
|
|||||||
|
# Resource Usage Monitoring - VisionFlow Automator
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
The VisionFlow Automator now includes real-time resource usage monitoring displayed directly in the UI. This feature helps users monitor the application's performance and system impact.
|
||||||
|
|
||||||
|
## Features Added
|
||||||
|
|
||||||
|
### 🔍 **Resource Usage Display**
|
||||||
|
A new "Resource Usage" group box has been added to the main UI showing:
|
||||||
|
|
||||||
|
#### Memory Usage
|
||||||
|
- **Process Memory**: Shows the application's memory usage in MB and percentage of system memory
|
||||||
|
- **Color-coded warnings**: Green for normal usage, red for high usage (>10% of system memory)
|
||||||
|
|
||||||
|
#### CPU Usage
|
||||||
|
- **Process CPU**: Shows the application's CPU usage percentage
|
||||||
|
- **Color-coded warnings**: Green for normal usage, red for high usage (>50%)
|
||||||
|
|
||||||
|
#### System Information
|
||||||
|
- **System Memory**: Shows overall system memory usage percentage and available memory in GB
|
||||||
|
- **Color-coded warnings**: Green for normal, red when system memory usage exceeds 85%
|
||||||
|
|
||||||
|
#### Cache Information
|
||||||
|
- **Template Cache**: Number of cached CV2 templates
|
||||||
|
- **Cooldown Entries**: Number of active step cooldowns
|
||||||
|
- **Color-coded status**: Blue for normal, orange when template cache exceeds 20 entries
|
||||||
|
|
||||||
|
#### Performance Metrics
|
||||||
|
- **Loop Time**: Average automation loop execution time in milliseconds
|
||||||
|
- **Color-coded performance**: Green for fast loops (<500ms), red for slow loops (>500ms)
|
||||||
|
|
||||||
|
### 🎛️ **Controls**
|
||||||
|
|
||||||
|
#### Toggle Button
|
||||||
|
- **Hide/Show Resources**: Button to toggle the visibility of resource usage information
|
||||||
|
- **Space-saving**: Users can hide the resource display if they don't need it
|
||||||
|
- **Tooltip**: Provides information about psutil installation for enhanced monitoring
|
||||||
|
|
||||||
|
#### Interactive Elements
|
||||||
|
- **Clickable Labels**: When psutil is not installed, clicking on resource labels shows installation instructions
|
||||||
|
- **Installation Guide**: Built-in dialog explaining how to install psutil for enhanced monitoring
|
||||||
|
|
||||||
|
### 📊 **Monitoring Levels**
|
||||||
|
|
||||||
|
#### Basic Monitoring (Without psutil)
|
||||||
|
When psutil is not installed, the display shows:
|
||||||
|
- Template cache size
|
||||||
|
- Cooldown entries count
|
||||||
|
- Basic performance metrics
|
||||||
|
- "N/A" for system metrics with installation instructions
|
||||||
|
|
||||||
|
#### Enhanced Monitoring (With psutil)
|
||||||
|
When psutil is installed (`pip install psutil`), the display shows:
|
||||||
|
- Detailed process memory usage
|
||||||
|
- CPU usage percentage
|
||||||
|
- System memory statistics
|
||||||
|
- Available system memory
|
||||||
|
- Process performance metrics
|
||||||
|
|
||||||
|
### ⚙️ **Technical Details**
|
||||||
|
|
||||||
|
#### Update Frequency
|
||||||
|
- **Real-time Updates**: Resource display updates every 2 seconds
|
||||||
|
- **Responsive UI**: Non-blocking updates that don't interfere with automation
|
||||||
|
- **Performance Optimized**: Minimal overhead from monitoring
|
||||||
|
|
||||||
|
#### Color Coding System
|
||||||
|
- **🟢 Green**: Normal/Good performance
|
||||||
|
- **🟡 Orange/Yellow**: Warning levels
|
||||||
|
- **🔴 Red**: High usage/Performance issues
|
||||||
|
- **⚫ Gray**: Unavailable/Error states
|
||||||
|
|
||||||
|
#### Error Handling
|
||||||
|
- **Graceful Degradation**: Shows basic info if detailed monitoring fails
|
||||||
|
- **Error Display**: Shows error messages for troubleshooting
|
||||||
|
- **Fallback Information**: Always shows cache and performance data
|
||||||
|
|
||||||
|
### 🎯 **Benefits**
|
||||||
|
|
||||||
|
#### For Users
|
||||||
|
- **Real-time Monitoring**: See exactly how much resources the app is using
|
||||||
|
- **Performance Insights**: Identify when the app is running slowly
|
||||||
|
- **System Impact**: Understand the app's impact on overall system performance
|
||||||
|
- **Troubleshooting**: Quickly identify memory leaks or performance issues
|
||||||
|
|
||||||
|
#### For Developers
|
||||||
|
- **Performance Metrics**: Built-in performance profiling
|
||||||
|
- **Memory Leak Detection**: Monitor memory usage over time
|
||||||
|
- **Cache Optimization**: Track template cache efficiency
|
||||||
|
- **System Resources**: Monitor system-wide impact
|
||||||
|
|
||||||
|
### 📋 **Usage Instructions**
|
||||||
|
|
||||||
|
#### Basic Usage
|
||||||
|
1. **View Resources**: Resource usage is displayed by default in the main window
|
||||||
|
2. **Toggle Display**: Click "Hide Resources" to save screen space
|
||||||
|
3. **Monitor Performance**: Watch for color changes indicating performance issues
|
||||||
|
|
||||||
|
#### Enhanced Monitoring Setup
|
||||||
|
1. **Install psutil**: Run `pip install psutil` in your Python environment
|
||||||
|
2. **Restart Application**: Close and reopen VisionFlow Automator
|
||||||
|
3. **Enhanced Display**: All metrics will now show detailed system information
|
||||||
|
|
||||||
|
#### Interpreting Metrics
|
||||||
|
- **Memory < 100MB**: Normal usage for typical scenarios
|
||||||
|
- **CPU < 20%**: Normal usage during automation
|
||||||
|
- **Loop Time < 200ms**: Good performance
|
||||||
|
- **Cache < 20 entries**: Normal template usage
|
||||||
|
|
||||||
|
### 🚨 **Warning Thresholds**
|
||||||
|
|
||||||
|
#### Memory Warnings
|
||||||
|
- **Process Memory > 10%**: High memory usage warning
|
||||||
|
- **System Memory > 85%**: System memory critical
|
||||||
|
|
||||||
|
#### Performance Warnings
|
||||||
|
- **CPU > 50%**: High CPU usage
|
||||||
|
- **Loop Time > 500ms**: Performance degradation
|
||||||
|
- **Cache > 20 entries**: Large template cache
|
||||||
|
|
||||||
|
### 🔧 **Troubleshooting**
|
||||||
|
|
||||||
|
#### High Memory Usage
|
||||||
|
- Check for memory leaks in long-running sessions
|
||||||
|
- Clear template cache periodically
|
||||||
|
- Reduce number of simultaneous image templates
|
||||||
|
|
||||||
|
#### High CPU Usage
|
||||||
|
- Reduce automation frequency
|
||||||
|
- Optimize image template sizes
|
||||||
|
- Check for infinite loops in scenarios
|
||||||
|
|
||||||
|
#### Slow Performance
|
||||||
|
- Monitor loop times for bottlenecks
|
||||||
|
- Reduce screenshot frequency
|
||||||
|
- Optimize template matching regions
|
||||||
|
|
||||||
|
#### Missing psutil
|
||||||
|
- Install with: `pip install psutil`
|
||||||
|
- Click on gray resource labels for installation instructions
|
||||||
|
- Restart application after installation
|
||||||
|
|
||||||
|
The resource monitoring feature provides valuable insights into the application's performance and helps users optimize their automation scenarios for better efficiency.
|
||||||
Binary file not shown.
@@ -5,3 +5,4 @@ pynput
|
|||||||
pygetwindow
|
pygetwindow
|
||||||
numpy
|
numpy
|
||||||
Pillow
|
Pillow
|
||||||
|
psutil # Optional: For enhanced resource monitoring
|
||||||
Reference in New Issue
Block a user