Rafi Ibn Sultan

I build vision–language systems that segment, ground, and reason about the world.

Ph.D. Candidate, Wayne State University Detroit, Michigan rafis@wayne.edu

On the academic job market. I'm seeking tenure-track Assistant Professor and Research Scientist positions, starting after I complete my Ph.D. in August 2027. Get in touch — I'd be glad to talk.

I'm a Ph.D. candidate in Computer Science at Wayne State University, working at the intersection of computer vision and multimodal AI. Since joining the Trustworthy AI Lab in 2022, I've worked on vision–language models and segmentation foundation models across medical imaging, mobility infrastructure, and remote sensing.

What I care about most is AI for social good — systems that don't just describe a scene, but reason about it well enough to support real decisions.

Curriculum Vitae Google Scholar GitHub
307 Citations 8 h-index 8 i10-index

Per Google Scholar, August 2026.

Research

What I work on

Grounded Multimodal Reasoning for Accessibility

My most recent work, WalkGPT (CVPR 2026), is a grounded vision–language model for pedestrian navigation and accessibility. It combines segmentation, depth estimation, and language reasoning to identify sidewalks, crosswalks, curb ramps, and accessibility barriers from real-world pedestrian-view imagery, then delivers step-by-step, context-aware guidance as a conversational agent.

Earlier in this line, GeoSAM (ECAI 2025) introduced sparse- and dense-prompt fine-tuning of SAM for large-scale mobility-infrastructure segmentation from aerial and street-level imagery. It has since been adopted as a benchmark in subsequent NeurIPS work.

I'm currently extending this toward stronger spatial reasoning in vision–language models at scale. SpatialCORE (under review) targets spatially confident reasoning in large VLMs — pushing past surface-level description toward predictions that are grounded in visual evidence rather than asserted.

Medical Image Segmentation & Multimodal Learning

I develop foundation-model-based segmentation and VLM-driven fusion methods for CT and MRI analysis. My current work in this line, MedPlex, explores deep vision–language co-adaptation for clinically grounded medical segmentation — building on BiPVL-Seg, which introduced bidirectional progressive alignment of visual features with structured medical text to improve organ and tumor segmentation.

collaborated with Henry Ford Health, I work on Left Anterior Descending (LAD) artery segmentation using novel encoder–decoder architectures — important for radiotherapy planning, since the LAD is highly sensitive to radiation injury. That work was presented at AAPM 2025 and is forthcoming in Medical Physics.

Technical Skills

Programming & Systems
  • Python
  • PyTorch
  • Linux
  • Git
  • LaTeX
  • Multi-GPU training
Models & Methods
  • Vision transformers
  • SAM
  • Large vision–language models
  • LoRA / PEFT
  • Contrastive learning
  • Segmentation
  • Grounding
  • Representation learning
Tools & Libraries
  • HuggingFace
  • MONAI
  • OpenCV
  • scikit-learn
  • Weights & Biases
Publications

Selected work

All Peer-Reviewed Publications

Preprints & Under Review

Full list on Google Scholar.

Talks & Service

Talks, coverage & recognition

Invited Talk
WalkGPT: Pixel-Grounded Navigation Guidance for Pedestrians
Best of CVPR 2026 Series, Voxel51 · July 2026 (virtual)
Invited Talk
WalkGPT: Grounded Vision–Language Conversation with Depth-Aware Segmentation for Pedestrian Navigation
Computer Vision Group, Cohere Labs Open Science Community · May 2026 (virtual)
Invited Talk
Multimodal AI for Mobility Infrastructure and Pedestrian Guidance
Departmental Seminar, Wayne State University · January 2026
Press
Detroit PBS: Wayne State Creates AI for Mobility Project
Feature on our lab and GeoSAM · April 2024
 Conference Talk
AutoProSAM presentation at WACV 2025
Tucson, Arizona · February 2025

Awards & Honors

  • Outstanding Graduate Research Assistant Award 2025
    Department of Computer Science, Wayne State University · Presented April 2026

Conference Reviewing

  • AAAI 2027
  • NeurIPS 2026
  • IJCAI-ECAI 2026
  • BMVC 2026
  • IJCNN 2025
  • ECCV

Journal Reviewing

News

Recent updates

  • Sep 4, 2026 Our work with Henry Ford Hospital, "A Neighborhood Attention Transformer Network for Enhanced 3D Segmentation of the Left Anterior Descending Artery," is now officially published in Medical Physics. Read it here, or the open-access version on arXiv.
  • Aug 7, 2026 Our paper "MedPlex: Deep Vision-Language Co-Adaptation for Clinically Grounded Medical Segmentation" has been accepted to the British Machine Vision Conference (BMVC 2026), taking place 23–26 November 2026 in Lancaster, UK.
  • Aug 4, 2026 Our paper with Henry Ford Hospital, "A Neighborhood Attention Transformer Network for Enhanced 3D Segmentation of the Left Anterior Descending Artery," has been accepted to Medical Physics.
  • Apr 16, 2026 Received the Outstanding Graduate Research Assistant Award for 2025 from the Department of Computer Science at Wayne State University.
  • Feb 20, 2026 WalkGPT has been accepted to CVPR 2026.
  • Jan 27, 2026 I was invited as a guest graduate speaker in the Computer Science Department at Wayne State University, presenting "From Mobility Infrastructure Segmentation to Pedestrian Guidance: A Multimodal AI Perspective."
  • Jul 11, 2025 GeoSAM has been accepted to the 28th European Conference on Artificial Intelligence (ECAI 2025) in Bologna, Italy.
Experience

Where I've worked

  • Graduate Research Assistant Sep 2023 — Present
    Trustworthy AI Lab, Department of Computer Science, Wayne State University · Detroit, MI
    • Lead independent research on grounded vision–language models for spatial reasoning, pedestrian navigation, and medical image segmentation — from problem formulation through publication, resulting in first-author papers at CVPR and ECAI.
    • Drive an ongoing clinical collaboration with Henry Ford Hospital on coronary CT segmentation, translating clinician requirements into model design and working directly with radiology partners on data, evaluation, and deployment-relevant validation.
    • Mentor junior Ph.D. students on research methodology, experimental design, and paper writing.
  • Graduate Teaching Assistant Aug 2022 — Aug 2023
    Department of Computer Science, Wayne State University · Detroit, MI
    • Led Software Engineering labs, supervising 5–6 project teams of 4–5 students each per term through the full development lifecycle, from requirements to delivery.
  • Lecturer Oct 2019 — Aug 2022
    Department of Computer Science and Engineering, Varendra University · Rajshahi, Bangladesh
    • Sole instructor for undergraduate courses including Microprocessor and Assembly Language, Object-Oriented Programming, and Computer Fundamentals — 30–40 students per course.
    • Supervised roughly 8 undergraduate final-year thesis and capstone projects from proposal to defense.
Education

Education

  • Ph.D. in Computer Science Sep 2022 — Expected Aug 2027
    Wayne State University · Detroit, MI
    Advisor: Prof. Dongxiao Zhu
  • M.Sc. in Computer Science Dec 2025
    Wayne State University · Detroit, MI
  • B.Sc. in Computer Science & Engineering 2014 — 2018
    Rajshahi University of Engineering & Technology (RUET) · Rajshahi, Bangladesh
  • Higher Secondary School Certificate 2013
    Rajshahi College
  • Secondary School Certificate 2011
    Shiroil Government High School
Beyond Research

Outside the lab

  • Football — a lifelong Real Madrid loyalist, and I'll defend Cristiano Ronaldo in any argument
  • Anime — always mid-series, always open to recommendations
  • Camping — weekends under canvas whenever the season allows
  • Hiking — trails are where I do my best thinking
  • Gaming — FIFA since '98, a Killjoy main in Valorant, and new to CS2
  • Movies & series — a reliable way to lose an evening
  • Beginner acoustic guitarist
  • Travel — working toward all 50 states

Conferences & Lab Life

Standing beside the WalkGPT poster at CVPR, which lays out the framework, the PAVE dataset construction, and qualitative navigation results.
WalkGPT at CVPR 2026Denver, Colorado
Members of the Trustworthy AI Lab gathered around a table of Detroit-style square pizza, raising their glasses toward the camera.
Pizza with the labDetroit, Michigan
Holding a Wayne State University research excellence certificate alongside two fellow award recipients and a faculty member.
Outstanding Graduate Research Assistant AwardWayne State University · 2025
Standing beside the AutoProSAM poster at WACV 2025, showing the architecture components and multi-organ segmentation results.
AutoProSAM at WACV 2025Tucson, Arizona
The lab group seated along a long table at a Detroit taproom, raising glasses for a Thanksgiving gathering.
Thanksgiving with the labDetroit, Michigan
The Trustworthy AI Lab gathered for a group photo in a backyard on the Fourth of July.
Fourth of July at Prof. Zhu'sTrustworthy AI Lab