@@ -0,0 +1,111 @@
|
||||
# VisionFlow Automator - Performance Optimizations
|
||||
|
||||
## Overview
|
||||
This document outlines the performance and memory optimizations implemented to prevent the application from being killed or stalled during scenario execution.
|
||||
|
||||
## Key Optimizations Implemented
|
||||
|
||||
### 1. Memory Management
|
||||
- **Template Caching**: Added `TemplateCache` class to cache loaded CV2 templates instead of reloading from disk every loop iteration
|
||||
- **Screenshot Caching**: Implemented screenshot caching with 50ms duration to reduce memory allocations
|
||||
- **Resource Cleanup**: Added proper cleanup in `cleanup_resources()` method and `closeEvent()`
|
||||
- **Garbage Collection**: Strategic `gc.collect()` calls to force memory cleanup at appropriate times
|
||||
- **Pixmap Memory Management**: Optimized image preview handling to prevent large pixmap memory leaks
|
||||
|
||||
### 2. Performance Improvements
|
||||
- **Optimized Loop Structure**: Redesigned automation loop to be more efficient
|
||||
- **Step Cooldown System**: Prevents rapid re-execution of the same step (1-second cooldown)
|
||||
- **Priority Execution**: Only executes one step per loop iteration to prevent blocking
|
||||
- **Dynamic Sleep**: Adjusts sleep time based on loop performance (0.05-0.2s)
|
||||
- **Performance Monitoring**: Added loop time tracking and warnings for slow performance
|
||||
|
||||
### 3. Threading Improvements
|
||||
- **Proper Thread Management**: Added timeout for thread joining in `stop_automation()`
|
||||
- **Background Processing**: Improved worker thread handling with better error recovery
|
||||
- **Non-blocking Operations**: Reduced blocking operations in the main thread
|
||||
|
||||
### 4. UI Optimizations
|
||||
- **Reduced Logging**: Changed default log level from DEBUG to INFO to reduce I/O overhead
|
||||
- **Efficient Image Previews**: Optimized image scaling and memory usage in dialogs
|
||||
- **State Update Throttling**: Reduced frequency of UI state updates
|
||||
|
||||
### 5. Resource Monitoring
|
||||
- **Performance Timer**: Added 10-second monitoring timer to track application health
|
||||
- **Memory Usage Tracking**: Optional psutil integration for detailed memory monitoring
|
||||
- **Cache Size Monitoring**: Tracks template cache size and warns when it grows large
|
||||
|
||||
### 6. Error Handling
|
||||
- **Graceful Degradation**: Better error handling in screenshot capture and template matching
|
||||
- **Resource Recovery**: Automatic cleanup on errors to prevent resource leaks
|
||||
- **Hotkey Error Handling**: Continues operation even if hotkey setup fails
|
||||
|
||||
## Configuration Changes
|
||||
|
||||
### PyAutoGUI Optimizations
|
||||
- Set `pyautogui.PAUSE = 0.01` (reduced from default 0.1s)
|
||||
- Set `pyautogui.MINIMUM_DURATION = 0` for faster actions
|
||||
- Kept `pyautogui.FAILSAFE = False` for automation reliability
|
||||
|
||||
### Screenshot Optimization
|
||||
- Added minimum size validation (10x10 pixels)
|
||||
- Improved tkinter screenshot tool with better UX
|
||||
- Memory-efficient PIL image handling with proper cleanup
|
||||
|
||||
### Template Matching
|
||||
- LRU cache with automatic cleanup of old entries
|
||||
- Maximum cache size of 50 templates
|
||||
- Force reload option for template updates
|
||||
|
||||
## Performance Targets
|
||||
|
||||
### Memory Usage
|
||||
- Template cache limited to 50 entries
|
||||
- Screenshot cache duration: 50ms
|
||||
- Automatic cleanup every 100 loop iterations
|
||||
- Garbage collection after major operations
|
||||
|
||||
### Timing
|
||||
- Target loop time: 50-200ms
|
||||
- Step cooldown: 1 second
|
||||
- Performance warning threshold: 1 second average loop time
|
||||
- Monitoring interval: 10 seconds
|
||||
|
||||
### Thread Safety
|
||||
- Worker thread timeout: 2 seconds
|
||||
- Daemon threads for automatic cleanup
|
||||
- Proper resource locking where needed
|
||||
|
||||
## Usage Recommendations
|
||||
|
||||
1. **Monitor Performance**: Check logs for performance warnings
|
||||
2. **Resource Management**: Regularly restart long-running sessions
|
||||
3. **Template Optimization**: Use appropriately sized template images
|
||||
4. **Step Design**: Avoid too many simultaneous image detections
|
||||
5. **System Resources**: Ensure adequate RAM for screenshot operations
|
||||
|
||||
## Future Improvements
|
||||
|
||||
1. **Multi-threading**: Consider separate threads for image processing
|
||||
2. **Image Compression**: Compress cached templates to save memory
|
||||
3. **Region Optimization**: Use smaller detection regions when possible
|
||||
4. **GPU Acceleration**: Consider OpenCV GPU operations for template matching
|
||||
5. **Background Processing**: Process non-critical operations in background
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### High Memory Usage
|
||||
- Check template cache size in logs
|
||||
- Reduce number of simultaneous steps
|
||||
- Restart application periodically
|
||||
|
||||
### Slow Performance
|
||||
- Check average loop times in logs
|
||||
- Reduce image template sizes
|
||||
- Simplify detection regions
|
||||
- Consider fewer simultaneous detections
|
||||
|
||||
### Application Crashes
|
||||
- Check log files for error patterns
|
||||
- Monitor system memory usage
|
||||
- Verify image file integrity
|
||||
- Check for corrupted templates
|
||||
@@ -0,0 +1,71 @@
|
||||
# VisionFlow Automator - Performance Optimization Summary
|
||||
|
||||
## Optimizations Applied
|
||||
|
||||
✅ **Memory Management**
|
||||
- Added template caching system to prevent repeated file I/O
|
||||
- Implemented screenshot caching with 50ms duration
|
||||
- Added proper resource cleanup and garbage collection
|
||||
- Optimized image preview handling to prevent memory leaks
|
||||
|
||||
✅ **Performance Improvements**
|
||||
- Redesigned automation loop for better efficiency
|
||||
- Added step cooldown system (1-second minimum between executions)
|
||||
- Implemented dynamic sleep timing based on performance
|
||||
- Added performance monitoring and warning system
|
||||
|
||||
✅ **Threading & Stability**
|
||||
- Improved worker thread management with proper cleanup
|
||||
- Added timeout handling for thread termination
|
||||
- Better error handling to prevent application crashes
|
||||
- Non-blocking operations in main UI thread
|
||||
|
||||
✅ **Resource Optimization**
|
||||
- Reduced logging verbosity to decrease I/O overhead
|
||||
- Optimized PyAutoGUI settings for faster execution
|
||||
- Improved screenshot selection tool with better UX
|
||||
- Added periodic cleanup every 100 loop iterations
|
||||
|
||||
## Key Features Added
|
||||
|
||||
### TemplateCache Class
|
||||
- LRU cache for CV2 template images
|
||||
- Automatic cleanup of old entries
|
||||
- Maximum 50 templates cached
|
||||
- Memory-efficient template loading
|
||||
|
||||
### Performance Monitoring
|
||||
- Loop time tracking and warnings
|
||||
- Memory usage monitoring
|
||||
- 10-second monitoring intervals
|
||||
- Automatic performance alerts
|
||||
|
||||
### Optimized Automation Loop
|
||||
- Priority-based step execution
|
||||
- Screenshot caching and reuse
|
||||
- Improved error handling and recovery
|
||||
- Dynamic timing adjustments
|
||||
|
||||
## Expected Results
|
||||
|
||||
- **Reduced Memory Usage**: Template caching and proper cleanup
|
||||
- **Faster Performance**: Optimized loops and reduced I/O
|
||||
- **Better Stability**: Improved error handling and resource management
|
||||
- **No More Stalling**: Non-blocking operations and better threading
|
||||
|
||||
## Configuration Changes
|
||||
|
||||
```python
|
||||
# PyAutoGUI optimizations
|
||||
pyautogui.PAUSE = 0.01 # Faster actions
|
||||
pyautogui.MINIMUM_DURATION = 0 # No minimum delays
|
||||
|
||||
# Logging optimization
|
||||
logging.basicConfig(level=logging.INFO) # Reduced verbosity
|
||||
|
||||
# Template cache settings
|
||||
max_cache_size = 50 # Maximum cached templates
|
||||
cache_cleanup_threshold = 25% # When to clean old entries
|
||||
```
|
||||
|
||||
The optimized application should now run more efficiently without getting killed or stalled during scenario execution.
|
||||
@@ -0,0 +1,90 @@
|
||||
# Resource Usage Display - Implementation Summary
|
||||
|
||||
## ✅ **Features Successfully Added**
|
||||
|
||||
### 🎯 **Main UI Enhancements**
|
||||
- **Resource Usage Group Box**: New section displaying real-time resource metrics
|
||||
- **Toggle Button**: Hide/Show resources button to save screen space
|
||||
- **Color-Coded Indicators**: Visual feedback for performance status
|
||||
- **Interactive Help**: Click to get psutil installation instructions
|
||||
|
||||
### 📊 **Monitoring Metrics**
|
||||
1. **Memory Usage**: Process memory in MB and percentage
|
||||
2. **CPU Usage**: Process CPU utilization percentage
|
||||
3. **System Memory**: Overall system memory usage and available space
|
||||
4. **Cache Information**: Template cache size and cooldown entries
|
||||
5. **Performance**: Average automation loop execution time
|
||||
|
||||
### 🔄 **Real-Time Updates**
|
||||
- **2-Second Refresh**: Resource display updates every 2 seconds
|
||||
- **Non-Blocking**: Updates don't interfere with automation performance
|
||||
- **Automatic Monitoring**: Starts immediately when application launches
|
||||
|
||||
### 🎨 **Visual Design**
|
||||
- **Color-Coded Status**:
|
||||
- 🟢 Green: Normal/Good performance
|
||||
- 🟡 Orange: Warning levels
|
||||
- 🔴 Red: High usage/Performance issues
|
||||
- ⚫ Gray: Unavailable metrics
|
||||
- **Compact Layout**: Fits seamlessly into existing UI
|
||||
- **Professional Styling**: Consistent with application theme
|
||||
|
||||
### 💡 **Smart Features**
|
||||
- **Fallback Mode**: Works without psutil, shows basic cache information
|
||||
- **Enhanced Mode**: Full system metrics when psutil is installed
|
||||
- **Error Handling**: Graceful degradation if monitoring fails
|
||||
- **Installation Guide**: Built-in help for psutil setup
|
||||
|
||||
## 🚀 **Benefits**
|
||||
|
||||
### For Performance Monitoring
|
||||
- **Real-Time Visibility**: See exactly how much resources the app uses
|
||||
- **Performance Bottlenecks**: Identify slow automation loops
|
||||
- **Memory Leak Detection**: Monitor memory usage over time
|
||||
- **System Impact**: Understand effect on overall system performance
|
||||
|
||||
### For Troubleshooting
|
||||
- **Quick Diagnosis**: Color-coded warnings for immediate issue identification
|
||||
- **Cache Monitoring**: Track template cache efficiency
|
||||
- **Resource Optimization**: Data to optimize scenario performance
|
||||
- **System Health**: Monitor system memory and CPU usage
|
||||
|
||||
## 📝 **Usage Instructions**
|
||||
|
||||
### Basic Usage
|
||||
1. **View Resources**: Resource panel is visible by default at the bottom of the main window
|
||||
2. **Toggle Display**: Click "Hide Resources" button to minimize screen usage
|
||||
3. **Monitor Status**: Watch color changes for performance alerts
|
||||
|
||||
### Enhanced Monitoring
|
||||
1. **Install psutil**: Run `pip install psutil` (already added to requirements.txt)
|
||||
2. **Restart App**: Close and reopen VisionFlow Automator
|
||||
3. **Full Metrics**: All detailed system information will be displayed
|
||||
|
||||
### Reading the Display
|
||||
- **Memory**: Shows process memory usage (green < 10%, red > 10%)
|
||||
- **CPU**: Shows process CPU usage (green < 50%, red > 50%)
|
||||
- **System**: Shows system memory usage (green < 85%, red > 85%)
|
||||
- **Cache**: Shows template cache size (blue normal, orange > 20 entries)
|
||||
- **Performance**: Shows loop execution time (green < 500ms, red > 500ms)
|
||||
|
||||
## 🔧 **Technical Implementation**
|
||||
|
||||
### Code Changes
|
||||
- Enhanced `get_memory_usage()` method with comprehensive metrics
|
||||
- Added `_update_resource_display()` for UI updates
|
||||
- Modified `_monitor_performance()` for real-time monitoring
|
||||
- Added resource display widgets to main UI layout
|
||||
- Implemented toggle functionality for space-saving
|
||||
|
||||
### Error Handling
|
||||
- Graceful fallback when psutil unavailable
|
||||
- Error display in resource panel for troubleshooting
|
||||
- Maintains basic functionality even if monitoring fails
|
||||
|
||||
### Performance Impact
|
||||
- Minimal overhead (2-second update cycle)
|
||||
- Non-blocking UI updates
|
||||
- Optimized memory usage for monitoring itself
|
||||
|
||||
The resource usage display provides valuable real-time insights into application performance and helps users optimize their automation scenarios for better efficiency and system resource management.
|
||||
@@ -0,0 +1,143 @@
|
||||
# Resource Usage Monitoring - VisionFlow Automator
|
||||
|
||||
## Overview
|
||||
The VisionFlow Automator now includes real-time resource usage monitoring displayed directly in the UI. This feature helps users monitor the application's performance and system impact.
|
||||
|
||||
## Features Added
|
||||
|
||||
### 🔍 **Resource Usage Display**
|
||||
A new "Resource Usage" group box has been added to the main UI showing:
|
||||
|
||||
#### Memory Usage
|
||||
- **Process Memory**: Shows the application's memory usage in MB and percentage of system memory
|
||||
- **Color-coded warnings**: Green for normal usage, red for high usage (>10% of system memory)
|
||||
|
||||
#### CPU Usage
|
||||
- **Process CPU**: Shows the application's CPU usage percentage
|
||||
- **Color-coded warnings**: Green for normal usage, red for high usage (>50%)
|
||||
|
||||
#### System Information
|
||||
- **System Memory**: Shows overall system memory usage percentage and available memory in GB
|
||||
- **Color-coded warnings**: Green for normal, red when system memory usage exceeds 85%
|
||||
|
||||
#### Cache Information
|
||||
- **Template Cache**: Number of cached CV2 templates
|
||||
- **Cooldown Entries**: Number of active step cooldowns
|
||||
- **Color-coded status**: Blue for normal, orange when template cache exceeds 20 entries
|
||||
|
||||
#### Performance Metrics
|
||||
- **Loop Time**: Average automation loop execution time in milliseconds
|
||||
- **Color-coded performance**: Green for fast loops (<500ms), red for slow loops (>500ms)
|
||||
|
||||
### 🎛️ **Controls**
|
||||
|
||||
#### Toggle Button
|
||||
- **Hide/Show Resources**: Button to toggle the visibility of resource usage information
|
||||
- **Space-saving**: Users can hide the resource display if they don't need it
|
||||
- **Tooltip**: Provides information about psutil installation for enhanced monitoring
|
||||
|
||||
#### Interactive Elements
|
||||
- **Clickable Labels**: When psutil is not installed, clicking on resource labels shows installation instructions
|
||||
- **Installation Guide**: Built-in dialog explaining how to install psutil for enhanced monitoring
|
||||
|
||||
### 📊 **Monitoring Levels**
|
||||
|
||||
#### Basic Monitoring (Without psutil)
|
||||
When psutil is not installed, the display shows:
|
||||
- Template cache size
|
||||
- Cooldown entries count
|
||||
- Basic performance metrics
|
||||
- "N/A" for system metrics with installation instructions
|
||||
|
||||
#### Enhanced Monitoring (With psutil)
|
||||
When psutil is installed (`pip install psutil`), the display shows:
|
||||
- Detailed process memory usage
|
||||
- CPU usage percentage
|
||||
- System memory statistics
|
||||
- Available system memory
|
||||
- Process performance metrics
|
||||
|
||||
### ⚙️ **Technical Details**
|
||||
|
||||
#### Update Frequency
|
||||
- **Real-time Updates**: Resource display updates every 2 seconds
|
||||
- **Responsive UI**: Non-blocking updates that don't interfere with automation
|
||||
- **Performance Optimized**: Minimal overhead from monitoring
|
||||
|
||||
#### Color Coding System
|
||||
- **🟢 Green**: Normal/Good performance
|
||||
- **🟡 Orange/Yellow**: Warning levels
|
||||
- **🔴 Red**: High usage/Performance issues
|
||||
- **⚫ Gray**: Unavailable/Error states
|
||||
|
||||
#### Error Handling
|
||||
- **Graceful Degradation**: Shows basic info if detailed monitoring fails
|
||||
- **Error Display**: Shows error messages for troubleshooting
|
||||
- **Fallback Information**: Always shows cache and performance data
|
||||
|
||||
### 🎯 **Benefits**
|
||||
|
||||
#### For Users
|
||||
- **Real-time Monitoring**: See exactly how much resources the app is using
|
||||
- **Performance Insights**: Identify when the app is running slowly
|
||||
- **System Impact**: Understand the app's impact on overall system performance
|
||||
- **Troubleshooting**: Quickly identify memory leaks or performance issues
|
||||
|
||||
#### For Developers
|
||||
- **Performance Metrics**: Built-in performance profiling
|
||||
- **Memory Leak Detection**: Monitor memory usage over time
|
||||
- **Cache Optimization**: Track template cache efficiency
|
||||
- **System Resources**: Monitor system-wide impact
|
||||
|
||||
### 📋 **Usage Instructions**
|
||||
|
||||
#### Basic Usage
|
||||
1. **View Resources**: Resource usage is displayed by default in the main window
|
||||
2. **Toggle Display**: Click "Hide Resources" to save screen space
|
||||
3. **Monitor Performance**: Watch for color changes indicating performance issues
|
||||
|
||||
#### Enhanced Monitoring Setup
|
||||
1. **Install psutil**: Run `pip install psutil` in your Python environment
|
||||
2. **Restart Application**: Close and reopen VisionFlow Automator
|
||||
3. **Enhanced Display**: All metrics will now show detailed system information
|
||||
|
||||
#### Interpreting Metrics
|
||||
- **Memory < 100MB**: Normal usage for typical scenarios
|
||||
- **CPU < 20%**: Normal usage during automation
|
||||
- **Loop Time < 200ms**: Good performance
|
||||
- **Cache < 20 entries**: Normal template usage
|
||||
|
||||
### 🚨 **Warning Thresholds**
|
||||
|
||||
#### Memory Warnings
|
||||
- **Process Memory > 10%**: High memory usage warning
|
||||
- **System Memory > 85%**: System memory critical
|
||||
|
||||
#### Performance Warnings
|
||||
- **CPU > 50%**: High CPU usage
|
||||
- **Loop Time > 500ms**: Performance degradation
|
||||
- **Cache > 20 entries**: Large template cache
|
||||
|
||||
### 🔧 **Troubleshooting**
|
||||
|
||||
#### High Memory Usage
|
||||
- Check for memory leaks in long-running sessions
|
||||
- Clear template cache periodically
|
||||
- Reduce number of simultaneous image templates
|
||||
|
||||
#### High CPU Usage
|
||||
- Reduce automation frequency
|
||||
- Optimize image template sizes
|
||||
- Check for infinite loops in scenarios
|
||||
|
||||
#### Slow Performance
|
||||
- Monitor loop times for bottlenecks
|
||||
- Reduce screenshot frequency
|
||||
- Optimize template matching regions
|
||||
|
||||
#### Missing psutil
|
||||
- Install with: `pip install psutil`
|
||||
- Click on gray resource labels for installation instructions
|
||||
- Restart application after installation
|
||||
|
||||
The resource monitoring feature provides valuable insights into the application's performance and helps users optimize their automation scenarios for better efficiency.
|
||||
Binary file not shown.
@@ -6,6 +6,9 @@ import time
|
||||
import logging
|
||||
import zipfile
|
||||
import shutil
|
||||
import gc
|
||||
import weakref
|
||||
from functools import lru_cache
|
||||
from PyQt6 import QtWidgets, QtGui, QtCore
|
||||
import cv2
|
||||
import pygetwindow as gw
|
||||
@@ -15,29 +18,83 @@ from pynput import keyboard
|
||||
import tkinter as tk
|
||||
from PIL import ImageGrab, Image, ImageTk
|
||||
|
||||
# Disable the PyAutoGUI fail-safe feature.
|
||||
# Disable the PyAutoGUI fail-safe feature and optimize settings
|
||||
pyautogui.FAILSAFE = False
|
||||
pyautogui.PAUSE = 0.01 # Reduce pause between actions
|
||||
pyautogui.MINIMUM_DURATION = 0 # Remove minimum duration for faster execution
|
||||
|
||||
# Setup logs directory
|
||||
LOGS_DIR = 'logs'
|
||||
if not os.path.exists(LOGS_DIR):
|
||||
os.makedirs(LOGS_DIR)
|
||||
|
||||
# Setup logging
|
||||
# Setup logging with optimized settings
|
||||
logging.basicConfig(
|
||||
level=logging.DEBUG,
|
||||
level=logging.INFO, # Changed from DEBUG to reduce log volume
|
||||
format='[%(asctime)s] %(levelname)s: %(message)s',
|
||||
handlers=[
|
||||
logging.FileHandler(os.path.join(LOGS_DIR, 'scenario_automation.log'), encoding='utf-8'),
|
||||
logging.StreamHandler()
|
||||
]
|
||||
)
|
||||
|
||||
# Create a more efficient logger for performance-critical sections
|
||||
perf_logger = logging.getLogger('performance')
|
||||
perf_logger.setLevel(logging.WARNING) # Only log warnings and errors for performance
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
CONFIG_DIR = 'scenarios'
|
||||
if not os.path.exists(CONFIG_DIR):
|
||||
os.makedirs(CONFIG_DIR)
|
||||
|
||||
class TemplateCache:
|
||||
"""
|
||||
Cache for template images to avoid reloading them repeatedly.
|
||||
Uses weak references to allow garbage collection when templates are no longer needed.
|
||||
"""
|
||||
def __init__(self, max_size=50):
|
||||
self._cache = {}
|
||||
self._max_size = max_size
|
||||
self._access_times = {}
|
||||
|
||||
def get_template(self, path, force_reload=False):
|
||||
"""Get template image from cache or load it."""
|
||||
if force_reload or path not in self._cache:
|
||||
if len(self._cache) >= self._max_size:
|
||||
self._cleanup_old_entries()
|
||||
|
||||
template = cv2.imread(path)
|
||||
if template is not None:
|
||||
self._cache[path] = template
|
||||
self._access_times[path] = time.time()
|
||||
return template
|
||||
else:
|
||||
self._access_times[path] = time.time()
|
||||
return self._cache[path]
|
||||
|
||||
def _cleanup_old_entries(self):
|
||||
"""Remove least recently used entries."""
|
||||
if not self._access_times:
|
||||
return
|
||||
|
||||
# Remove oldest 25% of entries
|
||||
sorted_entries = sorted(self._access_times.items(), key=lambda x: x[1])
|
||||
entries_to_remove = len(sorted_entries) // 4
|
||||
|
||||
for path, _ in sorted_entries[:entries_to_remove]:
|
||||
self._cache.pop(path, None)
|
||||
self._access_times.pop(path, None)
|
||||
|
||||
def clear(self):
|
||||
"""Clear all cached templates."""
|
||||
self._cache.clear()
|
||||
self._access_times.clear()
|
||||
gc.collect()
|
||||
|
||||
# Global template cache instance
|
||||
template_cache = TemplateCache()
|
||||
|
||||
class Scenario:
|
||||
"""
|
||||
Represents a scenario, which consists of a name and a list of steps.
|
||||
@@ -109,37 +166,52 @@ class Scenario:
|
||||
def take_screenshot_with_tkinter():
|
||||
"""
|
||||
Use Tkinter to let the user select a region of the screen for a screenshot.
|
||||
The screen is not frozen.
|
||||
Optimized for better performance and memory usage.
|
||||
Returns a dict with x, y, width, height, or None.
|
||||
"""
|
||||
root = tk.Tk()
|
||||
root.attributes("-alpha", 0.3)
|
||||
root.attributes("-fullscreen", True)
|
||||
root.attributes("-topmost", True) # Ensure window stays on top
|
||||
root.wait_visibility(root)
|
||||
|
||||
canvas = tk.Canvas(root, cursor="cross")
|
||||
# Configure for better performance
|
||||
root.resizable(False, False)
|
||||
root.overrideredirect(True)
|
||||
|
||||
canvas = tk.Canvas(root, cursor="cross", highlightthickness=0)
|
||||
canvas.pack(fill="both", expand=True)
|
||||
|
||||
rect = None
|
||||
start_x = None
|
||||
start_y = None
|
||||
|
||||
selection_rect = None
|
||||
|
||||
# Add instructions
|
||||
instruction_text = canvas.create_text(
|
||||
root.winfo_screenwidth() // 2, 50,
|
||||
text="Click and drag to select area. Press ESC to cancel.",
|
||||
fill="white", font=("Arial", 14)
|
||||
)
|
||||
|
||||
def on_button_press(event):
|
||||
nonlocal start_x, start_y, rect
|
||||
start_x = event.x
|
||||
start_y = event.y
|
||||
# Remove instruction text
|
||||
canvas.delete(instruction_text)
|
||||
rect = canvas.create_rectangle(start_x, start_y, start_x, start_y, outline='red', width=2)
|
||||
|
||||
def on_mouse_drag(event):
|
||||
nonlocal rect
|
||||
cur_x, cur_y = (event.x, event.y)
|
||||
if rect and start_x is not None and start_y is not None:
|
||||
cur_x, cur_y = event.x, event.y
|
||||
canvas.coords(rect, start_x, start_y, cur_x, cur_y)
|
||||
|
||||
def on_button_release(event):
|
||||
nonlocal selection_rect
|
||||
end_x, end_y = (event.x, event.y)
|
||||
if start_x is not None and start_y is not None:
|
||||
end_x, end_y = event.x, event.y
|
||||
|
||||
x1 = min(start_x, end_x)
|
||||
y1 = min(start_y, end_y)
|
||||
@@ -149,23 +221,44 @@ def take_screenshot_with_tkinter():
|
||||
width = x2 - x1
|
||||
height = y2 - y1
|
||||
|
||||
if width > 0 and height > 0:
|
||||
# Minimum size validation
|
||||
if width > 10 and height > 10:
|
||||
selection_rect = {"x": x1, "y": y1, "width": width, "height": height}
|
||||
else:
|
||||
selection_rect = None
|
||||
|
||||
root.quit()
|
||||
|
||||
canvas.bind("<ButtonPress-1>", on_button_press)
|
||||
canvas.bind("<B1-Motion>", on_mouse_drag)
|
||||
canvas.bind("<ButtonRelease-1>", on_button_release)
|
||||
|
||||
def on_escape(event):
|
||||
nonlocal selection_rect
|
||||
selection_rect = None
|
||||
root.quit()
|
||||
root.bind("<Escape>", on_escape)
|
||||
|
||||
def on_key_press(event):
|
||||
if event.keysym == 'Escape':
|
||||
on_escape(event)
|
||||
|
||||
# Bind events
|
||||
canvas.bind("<ButtonPress-1>", on_button_press)
|
||||
canvas.bind("<B1-Motion>", on_mouse_drag)
|
||||
canvas.bind("<ButtonRelease-1>", on_button_release)
|
||||
root.bind("<Escape>", on_escape)
|
||||
root.bind("<KeyPress>", on_key_press)
|
||||
|
||||
# Focus the window to receive key events
|
||||
root.focus_set()
|
||||
root.focus_force()
|
||||
|
||||
try:
|
||||
root.mainloop()
|
||||
except Exception as e:
|
||||
logger.error(f"Error in screenshot selection: {e}")
|
||||
selection_rect = None
|
||||
finally:
|
||||
try:
|
||||
root.destroy()
|
||||
except Exception as e:
|
||||
logger.warning(f"Error destroying tkinter window: {e}")
|
||||
|
||||
return selection_rect
|
||||
|
||||
@@ -176,82 +269,231 @@ class MainWindow(QtWidgets.QMainWindow):
|
||||
"""
|
||||
def automation_loop(self):
|
||||
"""
|
||||
Main loop for running automation steps. Continuously looks for image matches
|
||||
in the selected window or screen and performs actions as defined in the scenario.
|
||||
Optimized automation loop with memory management and performance improvements.
|
||||
"""
|
||||
logger.info('Automation loop started.')
|
||||
frame_count = 0
|
||||
last_gc_time = time.time()
|
||||
|
||||
try:
|
||||
while self.running:
|
||||
# Only update the state label, do not log every loop
|
||||
loop_start_time = time.time()
|
||||
|
||||
# Only update the state label periodically to reduce UI overhead
|
||||
if frame_count % 10 == 0:
|
||||
self.set_state('Looking for matches')
|
||||
|
||||
# Get optimized screenshot
|
||||
screen_data = self._get_optimized_screenshot()
|
||||
if screen_data is None:
|
||||
time.sleep(0.1)
|
||||
continue
|
||||
|
||||
screen_np, offset_x, offset_y = screen_data
|
||||
|
||||
# Process steps with priority and cooldown system
|
||||
step_executed = False
|
||||
current_time = time.time()
|
||||
|
||||
for step_idx, step in enumerate(self.current_scenario.steps):
|
||||
# Check step cooldown to prevent spam
|
||||
step_name = step.get('name', f'step_{step_idx}')
|
||||
if step_name in self._step_cooldown:
|
||||
if current_time - self._step_cooldown[step_name] < 1.0:
|
||||
continue
|
||||
|
||||
# Process step images with cache
|
||||
detections = self._process_step_images(step, screen_np, offset_x, offset_y)
|
||||
|
||||
# Check trigger condition
|
||||
if self._check_step_trigger(step, detections):
|
||||
self._processing_step = True
|
||||
self.set_state(f'Performing step: {step_name}')
|
||||
logger.info(f"Executing step: {step_name}")
|
||||
|
||||
# Execute step actions
|
||||
success = self._execute_step_actions(step, detections)
|
||||
|
||||
if success:
|
||||
self._step_cooldown[step_name] = current_time
|
||||
step_executed = True
|
||||
|
||||
self._processing_step = False
|
||||
break # Execute only one step per loop iteration
|
||||
|
||||
# Performance monitoring
|
||||
loop_time = time.time() - loop_start_time
|
||||
self._update_performance_stats(loop_time)
|
||||
|
||||
# Periodic cleanup
|
||||
frame_count += 1
|
||||
if frame_count % 100 == 0: # Every 100 frames
|
||||
self._periodic_cleanup()
|
||||
|
||||
# Dynamic sleep based on performance
|
||||
sleep_time = max(0.05, 0.2 - loop_time) if not step_executed else 0.1
|
||||
time.sleep(sleep_time)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f'Automation loop error: {e}')
|
||||
finally:
|
||||
self._processing_step = False
|
||||
self.set_state('Paused')
|
||||
logger.info('Automation loop ended.')
|
||||
|
||||
def _get_optimized_screenshot(self):
|
||||
"""
|
||||
Get screenshot with caching to reduce memory allocation.
|
||||
"""
|
||||
current_time = time.time()
|
||||
|
||||
# Use cached screenshot if recent enough and not processing
|
||||
if (self._last_screenshot is not None and
|
||||
not self._processing_step and
|
||||
current_time - self._last_screenshot_time < self._screenshot_cache_duration):
|
||||
return self._last_screenshot
|
||||
|
||||
try:
|
||||
selected_window = self.window_combo.currentText()
|
||||
if selected_window == 'Entire Screen':
|
||||
screen = pyautogui.screenshot()
|
||||
offset_x, offset_y = 0, 0
|
||||
else:
|
||||
try:
|
||||
# Try to get specific window
|
||||
win = None
|
||||
for w in gw.getAllWindows():
|
||||
if w.title == selected_window:
|
||||
win = w
|
||||
break
|
||||
|
||||
if win is not None and win.width > 0 and win.height > 0:
|
||||
x, y, w_, h_ = win.left, win.top, win.width, win.height
|
||||
# Validate window bounds
|
||||
if x >= 0 and y >= 0 and w_ > 0 and h_ > 0:
|
||||
screen = pyautogui.screenshot(region=(x, y, w_, h_))
|
||||
offset_x, offset_y = x, y
|
||||
else:
|
||||
screen = pyautogui.screenshot()
|
||||
offset_x, offset_y = 0, 0
|
||||
except Exception as e:
|
||||
logger.error(f'Error capturing window screenshot: {e}')
|
||||
else:
|
||||
screen = pyautogui.screenshot()
|
||||
offset_x, offset_y = 0, 0
|
||||
|
||||
# Convert to OpenCV format with optimized memory usage
|
||||
screen_np = cv2.cvtColor(np.array(screen), cv2.COLOR_RGB2BGR)
|
||||
for step in self.current_scenario.steps:
|
||||
# Detect all images for this step
|
||||
|
||||
# Cache the result
|
||||
screen_data = (screen_np, offset_x, offset_y)
|
||||
self._last_screenshot = screen_data
|
||||
self._last_screenshot_time = current_time
|
||||
|
||||
# Clean up PIL image to free memory
|
||||
del screen
|
||||
|
||||
return screen_data
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f'Error capturing screenshot: {e}')
|
||||
return None
|
||||
|
||||
def _process_step_images(self, step, screen_np, offset_x, offset_y):
|
||||
"""
|
||||
Process step images with template caching.
|
||||
"""
|
||||
detections = {}
|
||||
|
||||
for img in step.get('images', []):
|
||||
try:
|
||||
template = cv2.imread(img['path'])
|
||||
# Use cached template
|
||||
template = template_cache.get_template(img['path'])
|
||||
if template is None:
|
||||
logger.warning(f"Could not load template image: {img['path']}")
|
||||
continue
|
||||
|
||||
# Perform template matching with optimization
|
||||
res = cv2.matchTemplate(screen_np, template, cv2.TM_CCOEFF_NORMED)
|
||||
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(res)
|
||||
# logger.debug(f"Detection for {img['name']}: max_val={max_val}") # Suppressed per request
|
||||
|
||||
sensitivity = img.get('sensitivity', 0.9)
|
||||
if max_val > sensitivity:
|
||||
# Adjust detected location to be relative to the full screen
|
||||
abs_loc = (max_loc[0] + offset_x, max_loc[1] + offset_y)
|
||||
detections[img['name']] = (abs_loc, template.shape)
|
||||
detections[img['name']] = (abs_loc, template.shape, max_val)
|
||||
|
||||
# Clean up OpenCV result to free memory
|
||||
del res
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error in image detection for {img['name']}: {e}")
|
||||
# Check condition
|
||||
cond = step.get('condition', 'OR')
|
||||
found = [img for img in step.get('images', []) if img.get('name') in detections]
|
||||
trigger = False
|
||||
if not step.get('images', []):
|
||||
trigger = False
|
||||
elif cond == 'AND':
|
||||
trigger = len(found) == len(step.get('images', []))
|
||||
else:
|
||||
trigger = len(found) > 0
|
||||
# logger.debug(f"Step '{step.get('name', 'step')}' trigger check: found={found}, cond={cond}, trigger={trigger}") # Suppressed per request
|
||||
if trigger:
|
||||
self.set_state(f'Performing step: {step.get("name", "step")})')
|
||||
logger.info(f"Found match for step: {step.get('name', 'step')}. Performing actions.")
|
||||
# Use the first detected image for position
|
||||
ref_img = found[0] if found else step.get('images', [])[0]
|
||||
loc, shape = detections.get(ref_img.get('name'), ((0, 0), (0, 0, 0)))
|
||||
for act in step.get('actions', []):
|
||||
self._perform_step_action(act, loc, shape)
|
||||
logger.info(f"Performed actions for step: {step.get('name', 'step')}")
|
||||
time.sleep(1) # Prevent spamming
|
||||
time.sleep(0.2)
|
||||
logger.error(f"Error in image detection for {img.get('name', 'unknown')}: {e}")
|
||||
|
||||
return detections
|
||||
|
||||
def _check_step_trigger(self, step, detections):
|
||||
"""
|
||||
Check if step should be triggered based on detections.
|
||||
"""
|
||||
step_images = step.get('images', [])
|
||||
if not step_images:
|
||||
return False
|
||||
|
||||
found = [img for img in step_images if img.get('name') in detections]
|
||||
condition = step.get('condition', 'OR')
|
||||
|
||||
if condition == 'AND':
|
||||
return len(found) == len(step_images)
|
||||
else: # OR condition
|
||||
return len(found) > 0
|
||||
|
||||
def _execute_step_actions(self, step, detections):
|
||||
"""
|
||||
Execute step actions with error handling.
|
||||
"""
|
||||
try:
|
||||
found_images = [img for img in step.get('images', []) if img.get('name') in detections]
|
||||
if found_images:
|
||||
# Use the first detected image for position reference
|
||||
ref_img = found_images[0]
|
||||
loc, shape, confidence = detections.get(ref_img.get('name'), ((0, 0), (0, 0, 0), 0))
|
||||
|
||||
for action in step.get('actions', []):
|
||||
self._perform_step_action(action, loc, shape)
|
||||
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f'Automation loop error: {e}')
|
||||
finally:
|
||||
self.set_state('Paused')
|
||||
logger.error(f"Error executing step actions: {e}")
|
||||
|
||||
return False
|
||||
|
||||
def _update_performance_stats(self, loop_time):
|
||||
"""
|
||||
Update performance statistics for monitoring.
|
||||
"""
|
||||
self._loop_times.append(loop_time)
|
||||
if len(self._loop_times) > self._max_loop_time_samples:
|
||||
self._loop_times.pop(0)
|
||||
|
||||
# Log performance warnings
|
||||
avg_time = sum(self._loop_times) / len(self._loop_times)
|
||||
if avg_time > 0.5: # If average loop time exceeds 500ms
|
||||
logger.warning(f"Performance warning: Average loop time: {avg_time:.3f}s")
|
||||
|
||||
def _periodic_cleanup(self):
|
||||
"""
|
||||
Perform periodic cleanup to prevent memory leaks.
|
||||
"""
|
||||
# Clear old screenshot cache
|
||||
self._last_screenshot = None
|
||||
|
||||
# Clean up old step cooldowns (older than 5 minutes)
|
||||
current_time = time.time()
|
||||
old_cooldowns = [name for name, timestamp in self._step_cooldown.items()
|
||||
if current_time - timestamp > 300]
|
||||
for name in old_cooldowns:
|
||||
del self._step_cooldown[name]
|
||||
|
||||
# Force garbage collection
|
||||
gc.collect()
|
||||
|
||||
def _perform_step_action(self, action, loc, shape):
|
||||
"""
|
||||
@@ -319,33 +561,129 @@ class MainWindow(QtWidgets.QMainWindow):
|
||||
|
||||
def start_automation(self):
|
||||
"""
|
||||
Start the automation process and worker thread.
|
||||
Start the automation process and worker thread with resource checks.
|
||||
"""
|
||||
logger.info('Starting automation.')
|
||||
if not self.current_scenario or self.running:
|
||||
logger.warning('Start Automation: No scenario selected or already running.')
|
||||
return
|
||||
|
||||
# Perform pre-start cleanup
|
||||
self._last_screenshot = None
|
||||
self._step_cooldown.clear()
|
||||
gc.collect()
|
||||
|
||||
# Initialize performance monitoring
|
||||
self._loop_times.clear()
|
||||
|
||||
# Start automation
|
||||
self.running = True
|
||||
self.set_state('Looking for matches')
|
||||
self.btn_start_stop.setText(f'Stop ({self.hotkey.upper()})')
|
||||
|
||||
# Create and start worker thread
|
||||
self.worker = threading.Thread(target=self.automation_loop, daemon=True)
|
||||
self.worker.start()
|
||||
|
||||
# Setup hotkey listener with error handling
|
||||
try:
|
||||
self.listener = keyboard.GlobalHotKeys({self.hotkey: self.stop_automation})
|
||||
self.listener.start()
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to setup hotkey listener: {e}")
|
||||
# Continue without hotkey if setup fails
|
||||
|
||||
logger.info('Automation started successfully.')
|
||||
|
||||
def get_memory_usage(self):
|
||||
"""
|
||||
Get current memory usage information for monitoring.
|
||||
"""
|
||||
try:
|
||||
import psutil
|
||||
process = psutil.Process()
|
||||
memory_info = process.memory_info()
|
||||
cpu_percent = process.cpu_percent()
|
||||
|
||||
# Get system info
|
||||
system_memory = psutil.virtual_memory()
|
||||
|
||||
return {
|
||||
'process_memory_mb': memory_info.rss / 1024 / 1024, # MB
|
||||
'process_memory_percent': process.memory_percent(),
|
||||
'cpu_percent': cpu_percent,
|
||||
'system_memory_percent': system_memory.percent,
|
||||
'system_memory_available_gb': system_memory.available / 1024 / 1024 / 1024, # GB
|
||||
'template_cache_size': len(template_cache._cache) if hasattr(template_cache, '_cache') else 0,
|
||||
'cooldown_entries': len(self._step_cooldown) if hasattr(self, '_step_cooldown') else 0,
|
||||
'has_psutil': True
|
||||
}
|
||||
except ImportError:
|
||||
# psutil not available, return basic info
|
||||
return {
|
||||
'process_memory_mb': 0,
|
||||
'process_memory_percent': 0,
|
||||
'cpu_percent': 0,
|
||||
'system_memory_percent': 0,
|
||||
'system_memory_available_gb': 0,
|
||||
'template_cache_size': len(template_cache._cache) if hasattr(template_cache, '_cache') else 0,
|
||||
'cooldown_entries': len(self._step_cooldown) if hasattr(self, '_step_cooldown') else 0,
|
||||
'has_psutil': False
|
||||
}
|
||||
except Exception as e:
|
||||
logger.warning(f"Error getting memory usage: {e}")
|
||||
return {
|
||||
'process_memory_mb': 0,
|
||||
'process_memory_percent': 0,
|
||||
'cpu_percent': 0,
|
||||
'system_memory_percent': 0,
|
||||
'system_memory_available_gb': 0,
|
||||
'template_cache_size': len(template_cache._cache) if hasattr(template_cache, '_cache') else 0,
|
||||
'cooldown_entries': len(self._step_cooldown) if hasattr(self, '_step_cooldown') else 0,
|
||||
'has_psutil': False,
|
||||
'error': str(e)
|
||||
}
|
||||
|
||||
def stop_automation(self):
|
||||
"""
|
||||
Stop the automation process and worker thread.
|
||||
Stop the automation process and worker thread with proper cleanup.
|
||||
"""
|
||||
logger.info('Stopping automation.')
|
||||
self.running = False
|
||||
self.last_toggle_time = time.time()
|
||||
self.set_state('Paused')
|
||||
self.btn_start_stop.setText(f'Start ({self.hotkey.upper()})')
|
||||
|
||||
# Stop global hotkey listener
|
||||
if self.listener:
|
||||
try:
|
||||
self.listener.stop()
|
||||
except Exception as e:
|
||||
logger.warning(f"Error stopping hotkey listener: {e}")
|
||||
finally:
|
||||
self.listener = None
|
||||
|
||||
# Wait for worker thread to finish (with timeout)
|
||||
if self.worker and self.worker.is_alive():
|
||||
try:
|
||||
self.worker.join(timeout=2.0) # Wait up to 2 seconds
|
||||
if self.worker.is_alive():
|
||||
logger.warning("Worker thread did not stop within timeout")
|
||||
except Exception as e:
|
||||
logger.warning(f"Error joining worker thread: {e}")
|
||||
finally:
|
||||
self.worker = None
|
||||
|
||||
# Clear processing flags and cached data
|
||||
self._processing_step = False
|
||||
self._last_screenshot = None
|
||||
self._step_cooldown.clear()
|
||||
|
||||
# Force garbage collection to free memory
|
||||
gc.collect()
|
||||
|
||||
logger.info('Automation stopped and resources cleaned up.')
|
||||
|
||||
def __init__(self):
|
||||
"""
|
||||
Initialize the main window, UI, and load scenarios.
|
||||
@@ -364,9 +702,208 @@ class MainWindow(QtWidgets.QMainWindow):
|
||||
self.current_scenario = None
|
||||
self.selected_step_idx = None
|
||||
self.last_toggle_time = 0 # For debounce of start/stop
|
||||
|
||||
# Memory optimization attributes
|
||||
self._last_screenshot = None
|
||||
self._last_screenshot_time = 0
|
||||
self._screenshot_cache_duration = 0.05 # Cache screenshot for 50ms
|
||||
self._processing_step = False
|
||||
self._step_cooldown = {} # Cooldown tracking for steps
|
||||
|
||||
# Performance monitoring
|
||||
self._loop_times = []
|
||||
self._max_loop_time_samples = 10
|
||||
|
||||
self.init_ui()
|
||||
self.load_scenarios()
|
||||
|
||||
# Setup cleanup on close
|
||||
self.setAttribute(QtCore.Qt.WidgetAttribute.WA_DeleteOnClose)
|
||||
|
||||
# Setup periodic monitoring timer
|
||||
self.monitor_timer = QtCore.QTimer()
|
||||
self.monitor_timer.timeout.connect(self._monitor_performance)
|
||||
self.monitor_timer.start(2000) # Monitor every 2 seconds for responsive UI updates
|
||||
|
||||
# Initial resource display update
|
||||
QtCore.QTimer.singleShot(100, self._monitor_performance)
|
||||
|
||||
def _monitor_performance(self):
|
||||
"""
|
||||
Monitor application performance and memory usage.
|
||||
"""
|
||||
try:
|
||||
memory_info = self.get_memory_usage()
|
||||
|
||||
# Update resource display
|
||||
self._update_resource_display(memory_info)
|
||||
|
||||
# Log performance stats periodically (only if running)
|
||||
if self.running and hasattr(self, '_loop_times') and self._loop_times:
|
||||
avg_loop_time = sum(self._loop_times) / len(self._loop_times)
|
||||
|
||||
# Warning thresholds
|
||||
if avg_loop_time > 1.0:
|
||||
logger.warning(f"Performance issue: Average loop time {avg_loop_time:.3f}s")
|
||||
|
||||
# Memory warning for cache-based monitoring
|
||||
if memory_info.get('template_cache_size', 0) > 20:
|
||||
logger.info(f"Template cache has {memory_info['template_cache_size']} entries")
|
||||
|
||||
except Exception as e:
|
||||
logger.debug(f"Performance monitoring error: {e}")
|
||||
|
||||
def _update_resource_display(self, memory_info):
|
||||
"""
|
||||
Update the resource usage display in the UI.
|
||||
"""
|
||||
try:
|
||||
# Memory usage
|
||||
if memory_info.get('has_psutil', False):
|
||||
memory_text = f"Memory: {memory_info['process_memory_mb']:.1f}MB ({memory_info['process_memory_percent']:.1f}%)"
|
||||
memory_color = '#d9534f' if memory_info['process_memory_percent'] > 10 else '#5cb85c'
|
||||
else:
|
||||
memory_text = "Memory: N/A (psutil needed)"
|
||||
memory_color = '#f0ad4e'
|
||||
|
||||
self.memory_label.setText(memory_text)
|
||||
self.memory_label.setStyleSheet(f'font-size: 9pt; color: {memory_color};')
|
||||
|
||||
# Make memory label clickable to show psutil installation info
|
||||
if not memory_info.get('has_psutil', False):
|
||||
self.memory_label.mousePressEvent = lambda event: self._show_psutil_info()
|
||||
self.memory_label.setCursor(QtGui.QCursor(QtCore.Qt.CursorShape.PointingHandCursor))
|
||||
|
||||
# CPU usage
|
||||
if memory_info.get('has_psutil', False):
|
||||
cpu_text = f"CPU: {memory_info['cpu_percent']:.1f}%"
|
||||
cpu_color = '#d9534f' if memory_info['cpu_percent'] > 50 else '#5cb85c'
|
||||
else:
|
||||
cpu_text = "CPU: N/A"
|
||||
cpu_color = '#777'
|
||||
|
||||
self.cpu_label.setText(cpu_text)
|
||||
self.cpu_label.setStyleSheet(f'font-size: 9pt; color: {cpu_color};')
|
||||
|
||||
# System memory
|
||||
if memory_info.get('has_psutil', False):
|
||||
system_text = f"System: {memory_info['system_memory_percent']:.1f}% ({memory_info['system_memory_available_gb']:.1f}GB free)"
|
||||
system_color = '#d9534f' if memory_info['system_memory_percent'] > 85 else '#5cb85c'
|
||||
else:
|
||||
system_text = "System: N/A"
|
||||
system_color = '#777'
|
||||
|
||||
self.system_memory_label.setText(system_text)
|
||||
self.system_memory_label.setStyleSheet(f'font-size: 9pt; color: {system_color};')
|
||||
|
||||
# Cache info
|
||||
cache_text = f"Cache: {memory_info['template_cache_size']} templates, {memory_info['cooldown_entries']} cooldowns"
|
||||
cache_color = '#f0ad4e' if memory_info['template_cache_size'] > 20 else '#5bc0de'
|
||||
|
||||
self.cache_label.setText(cache_text)
|
||||
self.cache_label.setStyleSheet(f'font-size: 9pt; color: {cache_color};')
|
||||
|
||||
# Performance info
|
||||
if hasattr(self, '_loop_times') and self._loop_times:
|
||||
avg_time = sum(self._loop_times) / len(self._loop_times)
|
||||
perf_text = f"Performance: {avg_time*1000:.0f}ms avg loop time"
|
||||
perf_color = '#d9534f' if avg_time > 0.5 else '#5cb85c'
|
||||
else:
|
||||
perf_text = "Performance: Not running"
|
||||
perf_color = '#777'
|
||||
|
||||
self.performance_label.setText(perf_text)
|
||||
self.performance_label.setStyleSheet(f'font-size: 9pt; color: {perf_color};')
|
||||
|
||||
# Show error if any
|
||||
if 'error' in memory_info:
|
||||
self.performance_label.setText(f"Error: {memory_info['error'][:50]}...")
|
||||
self.performance_label.setStyleSheet('font-size: 9pt; color: #d9534f;')
|
||||
|
||||
except Exception as e:
|
||||
logger.debug(f"Error updating resource display: {e}")
|
||||
# Show basic fallback info
|
||||
self.memory_label.setText("Memory: Error")
|
||||
self.cpu_label.setText("CPU: Error")
|
||||
self.system_memory_label.setText("System: Error")
|
||||
self.cache_label.setText("Cache: Error")
|
||||
self.performance_label.setText(f"Display Error: {str(e)[:30]}...")
|
||||
|
||||
def stop_monitoring(self):
|
||||
"""Stop performance monitoring timer."""
|
||||
if hasattr(self, 'monitor_timer') and self.monitor_timer:
|
||||
self.monitor_timer.stop()
|
||||
|
||||
def _toggle_resource_display(self):
|
||||
"""Toggle the visibility of resource usage widgets."""
|
||||
if self.resource_widgets_visible:
|
||||
# Hide resource widgets
|
||||
self.memory_label.hide()
|
||||
self.cpu_label.hide()
|
||||
self.system_memory_label.hide()
|
||||
self.cache_label.hide()
|
||||
self.performance_label.hide()
|
||||
self.toggle_resources_btn.setText('Show Resources')
|
||||
self.resource_widgets_visible = False
|
||||
else:
|
||||
# Show resource widgets
|
||||
self.memory_label.show()
|
||||
self.cpu_label.show()
|
||||
self.system_memory_label.show()
|
||||
self.cache_label.show()
|
||||
self.performance_label.show()
|
||||
self.toggle_resources_btn.setText('Hide Resources')
|
||||
self.resource_widgets_visible = True
|
||||
|
||||
def _show_psutil_info(self):
|
||||
"""Show information about installing psutil for enhanced monitoring."""
|
||||
msg = QtWidgets.QMessageBox(self)
|
||||
msg.setWindowTitle("Enhanced Resource Monitoring")
|
||||
msg.setIcon(QtWidgets.QMessageBox.Icon.Information)
|
||||
msg.setText("Install psutil for detailed resource monitoring")
|
||||
msg.setInformativeText(
|
||||
"For detailed CPU, memory, and system resource monitoring, install the psutil package:\n\n"
|
||||
"pip install psutil\n\n"
|
||||
"This will enable:\n"
|
||||
"• Process memory usage in MB and percentage\n"
|
||||
"• CPU usage percentage\n"
|
||||
"• System memory statistics\n"
|
||||
"• Available system memory\n\n"
|
||||
"Without psutil, only basic cache information is shown."
|
||||
)
|
||||
msg.setStandardButtons(QtWidgets.QMessageBox.StandardButton.Ok)
|
||||
msg.exec()
|
||||
|
||||
def closeEvent(self, event):
|
||||
"""Handle application close event with proper cleanup."""
|
||||
self.cleanup_resources()
|
||||
event.accept()
|
||||
|
||||
def cleanup_resources(self):
|
||||
"""Clean up resources to prevent memory leaks."""
|
||||
if self.running:
|
||||
self.stop_automation()
|
||||
|
||||
# Stop monitoring timer
|
||||
self.stop_monitoring()
|
||||
|
||||
# Clear template cache
|
||||
template_cache.clear()
|
||||
|
||||
# Clear screenshot cache
|
||||
self._last_screenshot = None
|
||||
|
||||
# Clear performance tracking
|
||||
if hasattr(self, '_loop_times'):
|
||||
self._loop_times.clear()
|
||||
if hasattr(self, '_step_cooldown'):
|
||||
self._step_cooldown.clear()
|
||||
|
||||
# Force garbage collection
|
||||
gc.collect()
|
||||
|
||||
logger.info("Resources cleaned up successfully")
|
||||
|
||||
def init_ui(self):
|
||||
"""
|
||||
Initializes the main UI components and layouts.
|
||||
@@ -488,6 +1025,48 @@ class MainWindow(QtWidgets.QMainWindow):
|
||||
self.state_label.setStyleSheet('font-weight: bold; font-size: 11pt; color: #0055aa;')
|
||||
self.state_label.setAlignment(QtCore.Qt.AlignmentFlag.AlignLeft | QtCore.Qt.AlignmentFlag.AlignVCenter)
|
||||
|
||||
# Resource usage display
|
||||
self.resource_group = QtWidgets.QGroupBox('Resource Usage')
|
||||
self.resource_group.setSizePolicy(QtWidgets.QSizePolicy.Policy.Preferred, QtWidgets.QSizePolicy.Policy.Fixed)
|
||||
resource_layout = QtWidgets.QGridLayout()
|
||||
resource_layout.setSpacing(4)
|
||||
resource_layout.setContentsMargins(6, 6, 6, 6)
|
||||
|
||||
# Toggle button for resource display
|
||||
self.toggle_resources_btn = QtWidgets.QPushButton('Hide Resources')
|
||||
self.toggle_resources_btn.setMaximumWidth(100)
|
||||
self.toggle_resources_btn.setToolTip('Toggle resource usage display. Install psutil for detailed system metrics.')
|
||||
self.toggle_resources_btn.clicked.connect(self._toggle_resource_display)
|
||||
resource_layout.addWidget(self.toggle_resources_btn, 0, 2, 1, 1)
|
||||
|
||||
# Memory usage
|
||||
self.memory_label = QtWidgets.QLabel('Memory: --')
|
||||
self.memory_label.setStyleSheet('font-size: 9pt; color: #333;')
|
||||
resource_layout.addWidget(self.memory_label, 0, 0)
|
||||
|
||||
# CPU usage
|
||||
self.cpu_label = QtWidgets.QLabel('CPU: --')
|
||||
self.cpu_label.setStyleSheet('font-size: 9pt; color: #333;')
|
||||
resource_layout.addWidget(self.cpu_label, 0, 1)
|
||||
|
||||
# System memory
|
||||
self.system_memory_label = QtWidgets.QLabel('System: --')
|
||||
self.system_memory_label.setStyleSheet('font-size: 9pt; color: #333;')
|
||||
resource_layout.addWidget(self.system_memory_label, 1, 0)
|
||||
|
||||
# Cache info
|
||||
self.cache_label = QtWidgets.QLabel('Cache: --')
|
||||
self.cache_label.setStyleSheet('font-size: 9pt; color: #333;')
|
||||
resource_layout.addWidget(self.cache_label, 1, 1)
|
||||
|
||||
# Performance info
|
||||
self.performance_label = QtWidgets.QLabel('Performance: --')
|
||||
self.performance_label.setStyleSheet('font-size: 9pt; color: #333;')
|
||||
resource_layout.addWidget(self.performance_label, 2, 0, 1, 3)
|
||||
|
||||
self.resource_group.setLayout(resource_layout)
|
||||
self.resource_widgets_visible = True
|
||||
|
||||
# Main layout
|
||||
layout = QtWidgets.QGridLayout()
|
||||
layout.setHorizontalSpacing(6)
|
||||
@@ -500,6 +1079,10 @@ class MainWindow(QtWidgets.QMainWindow):
|
||||
layout.addWidget(window_group_box, 1, 0, 1, 5)
|
||||
layout.addWidget(steps_group_box, 2, 0, 3, 5)
|
||||
|
||||
# Resource usage group
|
||||
layout.addWidget(self.resource_group, 5, 0, 1, 5)
|
||||
layout.setRowStretch(5, 0) # Resource group should not stretch vertically
|
||||
|
||||
# Start/State row in a group
|
||||
start_state_group = QtWidgets.QGroupBox()
|
||||
start_state_group.setTitle("")
|
||||
@@ -510,8 +1093,8 @@ class MainWindow(QtWidgets.QMainWindow):
|
||||
start_state_layout.addWidget(self.state_label)
|
||||
start_state_layout.addStretch(1)
|
||||
start_state_group.setLayout(start_state_layout)
|
||||
layout.addWidget(start_state_group, 5, 0, 1, 5)
|
||||
layout.setRowStretch(5, 0) # Prevent vertical stretch
|
||||
layout.addWidget(start_state_group, 6, 0, 1, 5)
|
||||
layout.setRowStretch(6, 0) # Prevent vertical stretch
|
||||
|
||||
# Set central widget (must be at the end of init_ui)
|
||||
central = QtWidgets.QWidget()
|
||||
@@ -1208,6 +1791,7 @@ class StepDialog(QtWidgets.QDialog):
|
||||
def add_image_to_step(self):
|
||||
"""
|
||||
Add a new image to the step by taking a screenshot and letting the user select a region.
|
||||
Optimized for memory efficiency.
|
||||
"""
|
||||
main_window = self.parent()
|
||||
step_name = self.name_edit.text()
|
||||
@@ -1224,12 +1808,14 @@ class StepDialog(QtWidgets.QDialog):
|
||||
try:
|
||||
rect_coords = take_screenshot_with_tkinter()
|
||||
if rect_coords:
|
||||
cropped_pil_image = ImageGrab.grab(bbox=(
|
||||
# Use more memory-efficient screenshot capture
|
||||
bbox = (
|
||||
rect_coords['x'],
|
||||
rect_coords['y'],
|
||||
rect_coords['x'] + rect_coords['width'],
|
||||
rect_coords['y'] + rect_coords['height']
|
||||
))
|
||||
)
|
||||
cropped_pil_image = ImageGrab.grab(bbox=bbox)
|
||||
logger.debug("StepDialog.add_image_to_step: Screenshot selection finished.")
|
||||
except Exception as e:
|
||||
logger.error(f"StepDialog.add_image_to_step: Screenshot failed: {e}")
|
||||
@@ -1239,6 +1825,8 @@ class StepDialog(QtWidgets.QDialog):
|
||||
main_window.show()
|
||||
|
||||
if cropped_pil_image and rect_coords:
|
||||
try:
|
||||
# Convert PIL to QPixmap more efficiently
|
||||
img_np = np.array(cropped_pil_image.convert('RGB'))
|
||||
height, width, channel = img_np.shape
|
||||
bytes_per_line = 3 * width
|
||||
@@ -1257,24 +1845,75 @@ class StepDialog(QtWidgets.QDialog):
|
||||
safe_name = "".join(c for c in name if c.isalnum() or c in (' ', '_')).rstrip()
|
||||
path = os.path.join(step_images_dir, f'{safe_name}.png')
|
||||
|
||||
if cropped_pixmap.save(path, "PNG"):
|
||||
# Save with PNG optimization
|
||||
if cropped_pixmap.save(path, "PNG", quality=90):
|
||||
logger.info(f"Screenshot saved to {path}")
|
||||
img_obj = {'path': path, 'region': [rect_coords['x'], rect_coords['y'], rect_coords['width'], rect_coords['height']], 'name': name, 'sensitivity': 0.9}
|
||||
img_obj = {
|
||||
'path': path,
|
||||
'region': [rect_coords['x'], rect_coords['y'], rect_coords['width'], rect_coords['height']],
|
||||
'name': name,
|
||||
'sensitivity': 0.9
|
||||
}
|
||||
self.images.append(img_obj)
|
||||
self.img_list.addItem(name)
|
||||
else:
|
||||
logger.error(f"Failed to save screenshot to {path}")
|
||||
QtWidgets.QMessageBox.critical(self, 'Error', 'Failed to save the screenshot file.')
|
||||
|
||||
# Clean up memory
|
||||
del img_np, qimage, cropped_pixmap
|
||||
|
||||
finally:
|
||||
# Ensure PIL image is cleaned up
|
||||
if cropped_pil_image:
|
||||
cropped_pil_image.close()
|
||||
del cropped_pil_image
|
||||
gc.collect()
|
||||
|
||||
def update_image_preview(self, current, previous):
|
||||
"""
|
||||
Update the image preview label when a new image is selected.
|
||||
Optimized to prevent memory leaks from large pixmaps.
|
||||
"""
|
||||
# Clear previous pixmap to free memory
|
||||
if previous:
|
||||
self.img_preview.clear()
|
||||
|
||||
if current:
|
||||
try:
|
||||
idx = self.img_list.row(current)
|
||||
if idx < 0 or idx >= len(self.images):
|
||||
self.img_preview.setText('Image Preview')
|
||||
return
|
||||
|
||||
path = self.images[idx]['path']
|
||||
if not os.path.exists(path):
|
||||
self.img_preview.setText('Image Not Found')
|
||||
return
|
||||
|
||||
# Load and scale image efficiently
|
||||
pixmap = QtGui.QPixmap(path)
|
||||
self.img_preview.setPixmap(pixmap.scaled(self.img_preview.size(), QtCore.Qt.AspectRatioMode.KeepAspectRatio, QtCore.Qt.TransformationMode.SmoothTransformation))
|
||||
if pixmap.isNull():
|
||||
self.img_preview.setText('Invalid Image')
|
||||
return
|
||||
|
||||
# Scale with memory optimization
|
||||
preview_size = self.img_preview.size()
|
||||
if pixmap.width() > preview_size.width() or pixmap.height() > preview_size.height():
|
||||
scaled_pixmap = pixmap.scaled(
|
||||
preview_size,
|
||||
QtCore.Qt.AspectRatioMode.KeepAspectRatio,
|
||||
QtCore.Qt.TransformationMode.SmoothTransformation
|
||||
)
|
||||
self.img_preview.setPixmap(scaled_pixmap)
|
||||
# Clear original large pixmap
|
||||
del pixmap
|
||||
else:
|
||||
self.img_preview.setPixmap(pixmap)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error updating image preview: {e}")
|
||||
self.img_preview.setText('Preview Error')
|
||||
else:
|
||||
self.img_preview.setText('Image Preview')
|
||||
|
||||
|
||||
@@ -5,3 +5,4 @@ pynput
|
||||
pygetwindow
|
||||
numpy
|
||||
Pillow
|
||||
psutil # Optional: For enhanced resource monitoring
|
||||
Reference in New Issue
Block a user