Vision-language models (VLMs) can perform a broad range of tasks across diverse settings, yet their performance in agentic contexts remains poorly understood. Although online evaluations in simulators and real-world environments are essential for measuring interactive agent performance, they are costly and challenging to scale. Complementary offline VQA benchmarks can enable faster and more reliable comparisons across agentic domains, yet existing benchmarks remain limited and fragmented. To address this gap, we introduce AgentVQA, an offline benchmark for evaluating VLMs on realistic agentic scenarios across five agent domains: Web Agents, Robotics, Egocentric Videos, Games, and Spatial Planning. We build AgentVQA by manually selecting 1000 challenging, real-world agent examples from 23 source datasets and adapting them for reliable offline evaluation. The resulting benchmark preserves realistic scenarios from interactive environments across diverse domains, and its model rankings track those from corresponding online evaluations. Across the 22 evaluated models, the best achieves only 58.5% accuracy. Our cross-domain analysis shows that errors are driven primarily by failures to ground and maintain accurate representations of the current and evolving visual state, including localizing relevant UI elements, interpreting motion, tracking state changes over time, and inferring 3D spatial relationships, rather than by task misinterpretation or downstream logical reasoning.

Agentic frameworks show promise for automated editing but remain largely confined to unimodal workflows. Existing approaches struggle with complex, multi-reference instructions that span modalities, often resulting in grounding errors. To address this gap, we introduce M³-EDIT, an agentic framework for multi-modal, multi-reference, multi-task editing. M³-EDIT offers two key advantages: First, it can edit any target modality using reference inputs from any other modality, enabling rich cross-media editing. Second, it employs a two-stage architecture that decouples perception from action: an Analysis & Grounding stage first constructs a verified, machine-readable Unified Editing Canvas (UEC) over all inputs, and a separate Execution stage then operates on this canvas to carry out the requested edits. This design allows M³-Edit to reliably handle challenging tasks, such as targeted dialog replacement and visually-guided audio editing. We further introduce M³-BENCH, a benchmark comprising 110 complex editing tasks, where quantitative evaluations and human studies consistently validate the effectiveness of our framework.

We develop a cost-efficient neurosymbolic agent to address challenging multi-turn image editing tasks such as “Detect the bench in the image while recoloring it to pink. Also, remove the cat for a clearer view and recolor the wall to yellow.” It combines the fast, high-level subtask planning by large language models (LLMs) with the slow, accurate, tool-use, and local A^* search per subtask to find a cost-efficient toolpath—a sequence of calls to AI tools. To save the cost of A^* on similar subtasks, we perform inductive reasoning on previously successful toolpaths via LLMs to continuously extract/refine frequently used subroutines and reuse them as new tools for future tasks in an adaptive fast-slow planning, where the higher-level subroutines are explored first, and only when they fail, the low-level A^* search is activated. The reusable symbolic subroutines considerably save exploration cost on the same types of subtasks applied to similar images, yielding a human-like fast-slow toolpath agent “FaSTA^*”: fast subtask planning followed by rule-based subroutine selection per subtask is attempted by LLMs at first, which is expected to cover most tasks, while slow A^* search is only triggered for novel and challenging subtasks. By comparing with recent image editing approaches, we demonstrate FaSTA^* is significantly more computationally efficient while remaining competitive with the state-of-the-art baseline in terms of success rate. Our code and data can be accessed here.

Text-to-image models like stable diffusion and DALLE-3 still struggle with multi-turn image editing. We decompose such a task as an agentic workflow (path) of tool use that addresses a sequence of subtasks by AI tools of varying costs. Conventional search algorithms require expensive exploration to find tool paths. While large language models (LLMs) possess prior knowledge of subtask planning, they may lack accurate estimations of capabilities and costs of tools to determine which to apply in each subtask. Can we combine the strengths of both LLMs and graph search to find cost-efficient tool paths? We propose a three-stage approach “CoSTA” that leverages LLMs to create a subtask tree, which helps prune a graph of AI tools for the given task, and then conducts A search on the small subgraph to find a tool path. To better balance the total cost and quality, CoSTA* combines both metrics of each tool on every subtask to guide the A* search. Each subtask’s output is then evaluated by a vision-language model (VLM), where a failure will trigger an update of the tool’s cost and quality on the subtask. Hence, the A* search can recover from failures quickly to explore other paths. Moreover, CoSTA* can automatically switch between modalities across subtasks for a better cost-quality trade-off. We build a novel benchmark of challenging multi-turn image editing, on which CoSTA* outperforms state-of-the-art image-editing models or agents in terms of both cost and quality, and performs versatile trade-offs upon user preference.

Urban landscapes, emblematic of modernization and growth, are increasingly faced with the intricate challenge of visual pollution. This nuanced form of pollution, often overshadowed environmental discussions, profoundly influences the aesthetic harmony and mental well-being of urban inhabitants. In this research, we present an innovative methodology to detect visual pollution using drone-captured imagery. Our distinctive dataset captures a spectrum of visual pollutants, from graffiti, faded signage, and potholes to more complex issues like cluttered sidewalks and unkempt facades. Leveraging this dataset, we fine-tuned pre-trained object detection models, specifically YOLOv6, achieving remarkable accuracy in detecting these visual pollutants from images. Central to our study is the introduction of the Visual Pollution Index (VPI), a metric formulated through the multiplicative integration of the Counting Categories Ratio (CCR) and the Severity-Weighted Score (SWS). To provide a spatial representation of visual pollution levels, we further introduce heatmap visualizations. These heatmaps, overlaid on urban maps, offer a vivid depiction of pollution hotspots, enabling city planners and stakeholders to pinpoint areas of concern. Grounded in real-world perceptions, our approach offers a comprehensive lens to assess, visualize, and address visual pollution in urban environments.

In the rapidly evolving construction industry, timely and accurate monitoring of construction activities is paramount. This paper introduces a novel approach to quantifying construction activity using high-resolution drone imagery and advanced object detection models. We present a unique, high-quality dataset of construction site images captured from drones, which is distinct from the commonly used open-source or lower-quality datasets. Leveraging this dataset, we fine-tuned pre-trained object detection models to achieve remarkable accuracy in detecting construction materials from images. Building on this, we introduce the Composite Construction Activity Index (CAI), a metric formulated by combining the Material Diversity Ratio (MDR) and the Percentage of Image Covered by Construction Materials (PIC). This metric offers a comprehensive view of construction activity, capturing both its diversity and intensity. Additionally, we propose a methodology to map construction stages based on the detected materials, categorizing them into distinct phases of construction, providing insights into the project’s lifecycle. The paper culminates in the visualization of CAI values through heatmaps, offering a spatial representation of construction activity across cities or countries. Our findings and methodologies pave the way for more informed decision-making by stakeholders, from urban planners to construction managers, emphasizing the potential of drone technology and machine learning in revolutionizing construction monitoring.

Pneumonia is a serious lung infection that can be life-threatening if not diagnosed and treated on time. Accurately diagnosing pneumonia from X-ray images is essential for timely treatment and improved patient outcomes. In this research paper, we propose an approach for diagnosing pneumonia using X-ray images by leveraging the power of convolutional neural networks (CNNs) and transfer learning. The proposed approach utilizes a pre-trained MobileNet model, which is a popular CNN architecture, and adds additional layers on top to fine-tune the model for the specific task of diagnosing pneumonia. This approach was more effective than training a CNN model from scratch or using traditional machine-learning methods. Our proposed approach demonstrated superior performance when compared to previously utilized models and techniques as evidenced by the results obtained. The use of CNNs and transfer learning in medical imaging is a growing area of research, and our proposed approach is a significant advancement in the field of pneumonia diagnosis. The proposed approach has the potential to improve the accuracy and efficiency of pneumonia diagnosis and ultimately save lives.