CV
教育经历
东南大学 机械工程学院
2024.09 - 至今 机械工程 博士
- 导师:牛亚峰
2021.09 - 2024.06 设计学 硕士
中国农业大学 工学院
2017.08 - 2021.06 工业设计 本科
- GPA 3.9 / 4.0(专业排名 1/36,前 3%)
2018.09 - 2021.06 数据科学与大数据技术 双学位(信息与电气工程学院)
Southeast University, School of Mechanical Engineering
2024.09 - Present PhD in Mechanical Engineering
- Advisor: Yafeng Niu
2021.09 - 2024.06 MS in Design Science
China Agricultural University, College of Engineering
2017.08 - 2021.06 BS in Industrial Design
- GPA 3.9 / 4.0 (ranked 1/36, top 3% of major)
2018.09 - 2021.06 Dual degree in Data Science & Big Data Technology (College of Information and Electrical Engineering)
研究经历
融合实体控制器的眼控系统视触觉交互机制研究
2021.01 - 至今 参与 国家自然科学基金面上项目 72171044
- 基金申请书文献综述撰写、技术路线图绘制,成功申请
- 硕士课题:混合数字纸文本标绘眼触控交互研究(HoloLens 2, MRTK3)
- 凝视眨眼复合交互动作研究,论文被 Advanced Engineering Informatics 接收
- 眼动密码输入方法发明专利(已公开)
眼控装备系统视触觉交互机制研究
2023 - 2025 技术负责人 教育部联合基金
- 负责项目技术路线规划、实验设计与实施推进
面向眼控系统交互式界面元素的视觉表征与评价机制研究
2021.01 - 2021.12 参与 国家自然科学基金青年基金项目 71801037
- 研究平滑追踪眼控界面的控件尺寸和速度权衡,论文被 Displays (JCR Q1) 接收
震动晃动环境下的眼控交互界面设计
2022.01 - 2023.10 参与 173 计划技术领域基金项目
- 参与项目申请书撰写并成功申请,参与实验设计
眼动交互的隐式标定研究
2023.01 - 2023.10 参与 共用信息系统装备预研项目(基础类)
基于视听觉信息仿生认知的多模态人机交互技术
2021.09 - 2023.10 参与 航空科学基金一般项目 2020058069002
- 研究不同视听告警信息对航空操作员操作能力的影响,参与项目论证与实验设计
《眼控交互范式设计》专著编写
2024 - 2025 参与编写 / 技术统稿 科学出版社(已出版)
- 牛亚峰著,科学出版社 2025.12 出版,ISBN 978-7-03-083699-1,“智能人机交互前沿技术丛书”,全书约 30 万字
- 主笔第一章“绪论”(约 3 万字),系统建立眼控交互的知识体系;参与全书技术统稿与审阅
头戴式 NIR 眼动追踪系统(自研硬件平台)
2025 - 至今 硬件与算法负责人
- 双目近眼 NIR 相机 + 双路场景相机布局;整机约 50 g,前摄视场约 114°
- 近眼采样约 240 Hz,深度学习瞳孔检测约 60 fps,注视精度约 1–2°,并实现双目辐辏近距深度估计
- 在 LPW / TEyeD PitW / 自有 NIR 三个数据集上横向对比多组瞳孔检测模型,确定 YOLO 实时部署方案(自有 NIR 检测率约 93%,68 FPS)
- 对比 Linear / Poly2 / Poly3 / Vector 四种注视校准方法,最优 Poly3(RMSE 9.36 px)
- 完成 ArUco 屏幕校准 MVP 与 Streamlit 分析工具链(配置–采集–处理–结果全闭环)
Visual-Haptic Interaction Mechanisms in Eye-Control Systems with Physical Controllers
2021.01 - Present Participant NSFC General Program 72171044
- Wrote literature review and drew technical roadmap for the grant proposal (successfully funded)
- Master’s thesis: Mixed digital-paper text annotation via eye-touch interaction (HoloLens 2, MRTK3)
- Gaze-blink compound interaction study; paper accepted by Advanced Engineering Informatics
- Eye-tracking password input invention patent (published)
Visual-Haptic Interaction Mechanisms in Eye-Control Equipment Systems
2023 - 2025 Technical Lead Ministry of Education Joint Fund
- Led technical roadmap planning, experiment design, and implementation
Visual Representation and Evaluation of Interactive Interface Elements for Eye-Control Systems
2021.01 - 2021.12 Participant NSFC Youth Fund 71801037
- Studied widget-size and speed trade-offs in smooth-pursuit eye-control interfaces; paper accepted by Displays (JCR Q1)
Eye-Control Interface Design under Vibration and Sway Conditions
2022.01 - 2023.10 Participant 173 Program Technology Domain Fund
- Contributed to the successful grant proposal and to experiment design
Implicit Calibration for Gaze Interaction
2023.01 - 2023.10 Participant Shared Information System Equipment Pre-research Project (fundamental)
Multimodal HCI Based on Bionic Cognition of Audio-Visual Information
2021.09 - 2023.10 Participant Aeronautical Science Foundation 2020058069002
- Studied how visual and auditory alarm information affects aviation operator performance; contributed to project analysis and experiment design
Book: "Eye-Control Interaction Paradigm Design"
2024 - 2025 Contributing Author / Technical Editor Science Press (published)
- By Yafeng Niu, Science Press, Dec 2025, ISBN 978-7-03-083699-1; part of the “Frontier Technologies in Intelligent HCI” series (~300,000 characters)
- Authored Chapter 1, “Introduction” (~30,000 characters), establishing the knowledge framework for eye-control interaction; contributed to technical editing of the full manuscript
Head-Mounted NIR Eye-Tracking System (in-house hardware platform)
2025 - Present Hardware and Algorithm Lead
- Binocular near-eye NIR cameras plus dual scene cameras; ~50 g total weight, ~114° front-camera FOV
- ~240 Hz near-eye sampling, ~60 fps deep-learning pupil detection, ~1–2° gaze accuracy, with binocular-vergence near-field depth estimation
- Benchmarked pupil-detection models across LPW / TEyeD PitW / in-house NIR datasets; selected YOLO for real-time deployment (~93% detection rate, 68 FPS on in-house NIR)
- Compared Linear / Poly2 / Poly3 / Vector gaze-calibration methods; Poly3 performed best (RMSE 9.36 px)
- Built an ArUco screen-calibration MVP and a Streamlit toolchain covering configure–collect–process–results
项目经历
- 手眼口多通道交互项目(技术负责人,2023.04–2023.08):基于 Tobii 5 开发眼控交互,输出凝视点坐标与头动状态;基于 Unity 打通工控机与装备控制系统的 UDP 通信;用 WBS 与甘特图协调各组进度
- 基于 UE4 的三维建模与交互设计(技术负责人,2023.01–2023.06):Blender / 3ds Max 制作装备与地形资产,UE4 蓝图实现导弹追踪与直升机侦查动画仿真
- 基于 VR 的月球基地任务仿真与交互研究(技术负责人,2022.09–2023.01):设计月球探索任务脚本与需求文档,制作舱外太阳能板维修与月表采壤模块;建立知识库并培训 5 名成员的 VR 开发能力
- 基于 VR 手势的指控沙盘交互研究(VR 开发,2022.07–2022.09):基于 Quest 2 开发点/直线/曲线三种标绘功能,以及场景移动缩放、手掌菜单
- 企业综合能源智慧调度控制平台界面与交互设计(小组长,2022.12–2023.02):建立项目组件库适配 5 个平台,累计制作 31 个页面
- 立式芦苇笋剥皮机设计(负责人,国家级大学生创新创业项目,2018.12–2020.03):负责项目管理、机械电子设计、样机制作与专利申请;搭建基于机器视觉的大小头分拣试验台(Arduino + OpenMV)与三自由度剥皮装置;获 4 项省部级竞赛奖项与专利授权
- 基于眼控的自闭症儿童注意力训练软件设计(本科毕业设计,2020.11–2021.06):结合凝视触发与平滑追踪设计训练游戏,并在北京市海淀区雨露嘉禾儿童康复训练中心完成可用性测试
- Hand-Eye-Voice Multimodal Interaction (Technical Lead, 2023.04–2023.08): built Tobii 5-based gaze interaction exposing gaze coordinates and head state; bridged an industrial PC and an equipment control system over UDP in Unity; coordinated sub-teams with WBS and Gantt planning
- UE4-Based 3D Modeling and Interaction Design (Technical Lead, 2023.01–2023.06): produced equipment and terrain assets in Blender / 3ds Max; built missile-tracking and helicopter-reconnaissance animation simulations with UE4 Blueprints
- VR Lunar Base Mission Simulation (Technical Lead, 2022.09–2023.01): authored mission scripts and requirements; built EVA solar-panel repair and regolith-sampling modules; set up a knowledge base and trained 5 team members in VR development
- VR Gesture-Based Command Sandtable Interaction (VR Developer, 2022.07–2022.09): implemented point, line, and curve annotation on Quest 2, plus scene pan/zoom and a palm menu
- Integrated Energy Dispatch Platform UI/Interaction Design (Team Lead, 2022.12–2023.02): created a shared component library covering 5 platforms; delivered 31 screens
- Vertical Reed-Shoot Peeling Machine (Principal Investigator, National Undergraduate Innovation Program, 2018.12–2020.03): led project management, mechatronic design, prototyping, and patent filing; built a machine-vision orientation-sorting testbed (Arduino + OpenMV) and a 3-DOF peeling rig; won 4 provincial/ministerial competition awards and a granted patent
- Gaze-Based Attention Training Software for Children with Autism (Undergraduate Thesis, 2020.11–2021.06): designed a training game using gaze triggering and smooth pursuit; usability-tested at the Yulu Jiahe Children’s Rehabilitation Center, Haidian District, Beijing
论文与专著
完整列表详见 论文成果 页面。
在审 / 在投(4 篇)
- Wang, Y. et al. Gaze2Foot: Foot Primitives and Stabilization Mechanisms for Gaze-Foot Target Selection Across Postures. ACM TOCHI(CCF-A),一作,在审
- Wang, Y. et al. SPMark: Improving Small-Target Text Selection with Smooth-Pursuit Gaze Interaction in Hybrid Paper–Digital Environments. Virtual Reality(Springer, SCI),一作,一审修回、评审中
- Wang, Y. et al. Gaze Localizes, the Head Confirms: Staged Head–Eye Coordination for Hands-Free Selection in Dense AR. IJHCI(SCI),一作,在审
- Gaze-Driven Topological Phase-Transition Metasurfaces. Opto-Electronic Advances,共同一作,大修
代表性已发表论文
- Tian, J.-Z., Wang, Y.-Y., Wu, J.-C., Liu, C., & Niu, Y.-F. (2026). Exploring coordination in Gaze-Hand cascaded Interaction: Designing robust solutions for Low-Precision eye tracking. Advanced Engineering Informatics, 74, 104766.(第二作者)
- Huang, W.-C., Song, Z.-H., Wang, Y.-Y., Fan, L.-H., & Niu, Y.-F. (2026). Take-A-Look: Enabling intuitive distant target interactions in industrial virtual reality applications via natural gestures. Computers in Industry, 181, 104545.(第三作者)
- 牛亚峰, 王以俨*, 黄炜驰, 范林涵. (2025). 基于设计工效学的眼控交互范式设计研究现状与进展. 《包装工程》, 46(4), 20-35.(北大核心、AMI 收录;第二作者兼通信作者)
- Huang, W.-C., Wang, Y.-Y., Fan, L.-H., Yang, S.-T., & Niu, Y.-F. (2025). Enhancing gaze interaction performance: Design and optimization of perspective-driven fisheye view. Advanced Engineering Informatics, 65, 103365.(第二作者)
- Wang, Y.-Y., Tian, J.-Z., Xiao, L., He, J.-X., & Niu, Y.-F. (2024). Research on a spatial–temporal characterisation of blink-triggered eye control interactions. Advanced Engineering Informatics, 59, 102297.(一作)
- Wang, Y., Wen, C., Tian, J., Huang, W.-C., Niu, Y., Yang, W., & Xue, C. (2024). Design recommendations of target size and tracking speed … for smooth pursuit in eye-control system. Displays, 81, 102608.(一作)
专著
- 牛亚峰. (2025). 《眼控交互范式设计》. 北京:科学出版社. ISBN 978-7-03-083699-1.(“智能人机交互前沿技术丛书”,全书约 30 万字;王以俨主笔第一章“绪论”约 3 万字,并参与全书技术统稿)
See the Publications page for the full list.
Under review / submitted (4)
- Wang, Y. et al. Gaze2Foot: Foot Primitives and Stabilization Mechanisms for Gaze-Foot Target Selection Across Postures. ACM TOCHI (CCF-A), first author, under review
- Wang, Y. et al. SPMark: Improving Small-Target Text Selection with Smooth-Pursuit Gaze Interaction in Hybrid Paper–Digital Environments. Virtual Reality (Springer, SCI), first author, revised and under review
- Wang, Y. et al. Gaze Localizes, the Head Confirms: Staged Head–Eye Coordination for Hands-Free Selection in Dense AR. IJHCI (SCI), first author, under review
- Gaze-Driven Topological Phase-Transition Metasurfaces. Opto-Electronic Advances, co-first author, major revision
Selected published papers
- Tian, J.-Z., Wang, Y.-Y., Wu, J.-C., Liu, C., & Niu, Y.-F. (2026). Exploring coordination in Gaze-Hand cascaded Interaction: Designing robust solutions for Low-Precision eye tracking. Advanced Engineering Informatics, 74, 104766. (second author)
- Huang, W.-C., Song, Z.-H., Wang, Y.-Y., Fan, L.-H., & Niu, Y.-F. (2026). Take-A-Look: Enabling intuitive distant target interactions in industrial virtual reality applications via natural gestures. Computers in Industry, 181, 104545. (third author)
- Y. Niu, Y. Wang*, W. Huang, L. Fan. (2025). Research status and progress of eye-control interaction paradigm design based on design ergonomics. Packaging Engineering, 46(4), 20-35. (PKU core journal, AMI-indexed; second author and corresponding author)
- Huang, W.-C., Wang, Y.-Y., Fan, L.-H., Yang, S.-T., & Niu, Y.-F. (2025). Enhancing gaze interaction performance: Design and optimization of perspective-driven fisheye view. Advanced Engineering Informatics, 65, 103365. (second author)
- Wang, Y.-Y., Tian, J.-Z., Xiao, L., He, J.-X., & Niu, Y.-F. (2024). Research on a spatial–temporal characterisation of blink-triggered eye control interactions. Advanced Engineering Informatics, 59, 102297. (first author)
- Wang, Y., Wen, C., Tian, J., Huang, W.-C., Niu, Y., Yang, W., & Xue, C. (2024). Design recommendations of target size and tracking speed … for smooth pursuit in eye-control system. Displays, 81, 102608. (first author)
Monograph
- Y. Niu. (2025). Eye-Control Interaction Paradigm Design. Beijing: Science Press. ISBN 978-7-03-083699-1. (Frontier Technologies in Intelligent HCI series, ~300,000 characters; Y. Wang authored Chapter 1, ~30,000 characters, and contributed to technical editing)
专利
眼控交互 / 眼动追踪(东南大学,8 项已公开)
- 牛亚峰, 王以俨, 蔡劼霖, 黄炜驰, 范林涵, 王铸涛. (2026). 近眼AR设备注视隐式自适应校准方法、系统、设备及介质. 申请号 CN202610582485.2,公开号 CN122431533A,公开日 2026-07-21.【发明专利】
- 牛亚峰, 王以俨, 王妍妍, 黄炜驰, 范林涵, 王铸涛. (2026). 近眼AR设备凝视深度歧义消解方法、交互设备及存储介质. 申请号 CN202610520767.X,公开号 CN122369009A,公开日 2026-07-10.【发明专利】
- 牛亚峰, 王以俨, 尹慧芳, 黄炜驰, 范林涵, 王铸涛. (2026). 多模态大模型交互消歧方法、系统、设备及介质(基于眼动意图模式分类). 申请号 CN202610621745.2,公开号 CN122286538A,公开日 2026-06-26.【发明专利】
- 牛亚峰, 范林涵, 黄炜驰, 王以俨, 王铸涛, 何花. (2026). 一种基于DMA的多通道超声换能器相控阵驱动方法. 申请号 CN202610009022.7,公开号 CN121763911A,公开日 2026-03-31.【发明专利】
- 牛亚峰, 黄炜驰, 王以俨, 范林涵, 王铸涛. (2026). 基于误差反向引导的眼动交互方法及交互系统. 申请号 CN202511546035.X,公开号 CN121349306A,公开日 2026-01-16.【发明专利】
- 牛亚峰, 王以俨, 范林涵, 黄炜驰. (2024). 混合凝视平滑追踪的眼触控小目标对象选择方法及设备. 申请号 CN202411140247.3,公开号 CN119045661A,公开日 2024-11-29.【发明专利】
- 牛亚峰, 王以俨, 黄炜驰, 范林涵. (2024). 基于平滑追踪的混合数字纸的文本标绘系统及标绘方法. 申请号 CN202411140022.8,公开号 CN119045660A,公开日 2024-11-29.【发明专利】
- 田景泽, 牛亚峰, 王以俨, 何家新, 黄炜驰, 杨文骏. (2023). 眼动密码输入方法、系统及设备. 申请号 CN202310365466.0,公开号 CN116755544A,公开日 2023-09-15.【发明专利】
另有 1 项已受理待公开:基于触觉运动幻觉诱发跨模态方向性响应的感知确认交互方法、系统、交互设备及存储介质(申请号 CN202610695594.5,2026-05-20 受理);2 项眼动追踪发明专利正在与代理修改中。
本科阶段(中国农业大学)
- 柳沙, 王以俨, 李琳, 李鸿盛, 谭宇. (2021). 一种自动化芦苇笋剥笋装置. 申请号 CN202021067017.6,授权公告号 CN212520711U.【实用新型,已授权】
- 柳沙, 龙杨, 李钰冰, 鞠雨欣, 薄佳乐, 杨凤, 王以俨. (2020). 立式芦苇笋剥笋装置. 申请号 CN201811180636.3,授权公告号 CN109222129B.【发明专利,已授权】
- 柳沙, 鞠雨欣, 李钰冰, 龙杨, 王以俨, 杨凤, 彭文博. (2019). 整杆式甘蔗打捆收获机. 申请号 CN201920095424.9,授权公告号 CN209861624U.【实用新型,已授权】
Eye-Control / Eye-Tracking (Southeast University, 8 published)
- Y. Niu, Y. Wang, J. Cai, W. Huang, L. Fan, Z. Wang. (2026). Implicit adaptive gaze calibration method, system, device, and medium for near-eye AR. App. No. CN202610582485.2, Pub. No. CN122431533A, 2026-07-21. [Invention Patent]
- Y. Niu, Y. Wang, Y. Wang, W. Huang, L. Fan, Z. Wang. (2026). Near-eye AR device gaze-depth ambiguity resolution method, interactive device, and storage medium. App. No. CN202610520767.X, Pub. No. CN122369009A, 2026-07-10. [Invention Patent]
- Y. Niu, Y. Wang, H. Yin, W. Huang, L. Fan, Z. Wang. (2026). Multimodal LLM interaction disambiguation method, system, device, and medium (gaze-intent pattern classification). App. No. CN202610621745.2, Pub. No. CN122286538A, 2026-06-26. [Invention Patent]
- Y. Niu, L. Fan, W. Huang, Y. Wang, Z. Wang, H. He. (2026). DMA-based multi-channel ultrasonic transducer phased-array driving method. App. No. CN202610009022.7, Pub. No. CN121763911A, 2026-03-31. [Invention Patent]
- Y. Niu, W. Huang, Y. Wang, L. Fan, Z. Wang. (2026). Error-reverse-guided gaze interaction method and interaction system. App. No. CN202511546035.X, Pub. No. CN121349306A, 2026-01-16. [Invention Patent]
- Y. Niu, Y. Wang, L. Fan, W. Huang. (2024). Hybrid gaze-smooth-pursuit eye-touch small-target selection method and device. App. No. CN202411140247.3, Pub. No. CN119045661A, 2024-11-29. [Invention Patent]
- Y. Niu, Y. Wang, W. Huang, L. Fan. (2024). Smooth-pursuit-based hybrid digital-paper text annotation system and method. App. No. CN202411140022.8, Pub. No. CN119045660A, 2024-11-29. [Invention Patent]
- J. Tian, Y. Niu, Y. Wang, J. He, W. Huang, W. Yang. (2023). Eye-tracking password input method, system, and device. App. No. CN202310365466.0, Pub. No. CN116755544A, 2023-09-15. [Invention Patent]
One further patent is filed and awaiting publication: perceptual-confirmation interaction based on tactile-motion-illusion-induced cross-modal directional response (App. No. CN202610695594.5, filed 2026-05-20). Two more eye-tracking patents are in revision with the patent agent.
Undergraduate Period (China Agricultural University)
- S. Liu, Y. Wang, L. Li, H. Li, Y. Tan. (2021). Automatic reed-shoot peeling device. App. No. CN202021067017.6, Grant No. CN212520711U. [Utility Model, Granted]
- S. Liu, Y. Long, Y. Li, Y. Ju, J. Bo, F. Yang, Y. Wang. (2020). Vertical reed-shoot peeling device. App. No. CN201811180636.3, Grant No. CN109222129B. [Invention Patent, Granted]
- S. Liu, Y. Ju, Y. Li, Y. Long, Y. Wang, F. Yang, W. Peng. (2019). Whole-stalk sugarcane bundling harvester. App. No. CN201920095424.9, Grant No. CN209861624U. [Utility Model, Granted]
软件著作权
- 殷成刚, 王以俨. (2020). 基于Python的工程信息交互仿真软件(V1.0). 中国农业大学.
- 殷成刚, 王以俨. (2020). 基于Python语言的新产品开发流程动画仿真软件(V1.0). 中国农业大学.
- 殷成刚, 王以俨. (2020). 基于Python语言多项目并行开发时间管理软件(V1.0). 中国农业大学.
- C. Yin, Y. Wang. (2020). Python-Based Engineering Information Communication Simulation Software (v1.0). China Agricultural University.
- C. Yin, Y. Wang. (2020). Python-Based New Product Development Process Animation Simulation Software (v1.0). China Agricultural University.
- C. Yin, Y. Wang. (2020). Python-Based Multi-Project Concurrent Development Time-Management Software (v1.0). China Agricultural University.
技能
- 眼动与 XR
- Pupil Labs、Tobii、自研 NIR 眼动仪;HoloLens 2、Unity、MRTK3、Vuforia
- 编程与算法
- Python(PyTorch、YOLO、OpenCV、PySide6)、C#、R
- 实验与统计
- 工效学实验设计、pandas / scipy / statsmodels、SPSS、质性研究
- 语言
- 中文(母语)、英语
- Eye tracking & XR
- Pupil Labs, Tobii, in-house NIR eye tracker; HoloLens 2, Unity, MRTK3, Vuforia
- Programming & algorithms
- Python (PyTorch, YOLO, OpenCV, PySide6), C#, R
- Experiments & statistics
- ergonomic experiment design, pandas / scipy / statsmodels, SPSS, qualitative methods
- Languages
- Chinese (native), English
荣誉与奖项
学术与荣誉称号
- 东南大学优秀硕士学位论文
- 北京市优秀毕业生(2021)
- 国家奖学金(两次:2018–2019、2019–2020)
学科竞赛
- 全国大学生工业设计大赛(北京分赛区)一等奖(2019)
- 全国大学生广告艺术大赛 三等奖(2020)
- 全国大学生广告艺术大赛(北京分赛区)二等奖(2019)
- 第五届全国大学生智能农业装备创新大赛 二等奖(2019)
- 首届中国(怀化)乡村振兴设计创新大赛 最佳绿色设计奖(2019)
- 北京市大学生工程设计表达竞赛 二等奖(2018)
Academic Honors
- Outstanding Master’s Thesis, Southeast University
- Outstanding Graduate of Beijing Municipality (2021)
- National Scholarship (twice: 2018–2019, 2019–2020)
Student Competitions
- First Prize, National Industrial Design Competition for College Students (Beijing Division), 2019
- Third Prize, National College Advertising Art Competition, 2020
- Second Prize, National College Advertising Art Competition (Beijing Division), 2019
- Second Prize, 5th National Intelligent Agricultural Equipment Innovation Competition, 2019
- Best Green Design Award, 1st China (Huaihua) Rural Revitalization Design Innovation Competition, 2019
- Second Prize, Beijing Collegiate Engineering Design Expression Competition, 2018
