Discussion and Conclusion, and References

Table of Links

Abstract and I. Introduction

II. Related Work

III. Kinematics of Traffic Agents

IV. Methodology

V. Results

VI. Discussion and Conclusion, and References

VII. Appendix

VI. DISCUSSION AND CONCLUSION

In this paper, we present a simple method for including kinematic relationships in probabilistic trajectory forecasting. Kinematic priors can also be implemented for deterministic methods where linear approximations are not necessary. With nearly no additional overhead, we not only show improvement in models trained on robust datasets but also in suboptimal settings with small datasets and noisy trajectories, with up to 12% improvement in smaller datasets and 1% less performance degradation in the presence of noise for the full Waymo dataset. For overall performance improvement, we find Formulation 1 with velocity components to be the most beneficial and well-rounded to prediction performance.

When there is large-scale data to learn a good model of how vehicles move, we observe that the effects of kinematic priors are less pronounced. This is demonstrated by the less obvious improvements over the baseline in Table I compared to Table II; model complexity and dataset size will eventually out-scale the effects of the kinematic prior. With enough resources and high-quality data, trajectory forecasting models will learn to “reinvent the steering wheel”, or implicitly learn how vehicles move via the complexity of the neural network.

One limitation is that we primarily explore analysis in one-shot prediction; future work focusing on kinematic priors for autoregressive approaches would be interesting in comparison to one-shot models with kinematic priors, especially since autoregressive approaches model predictions conditionally based on previous timesteps.

In future work, kinematic priors can be further explored for transfer learning between domains. While distributions of trajectories may change in scale and distribution depending on the environment, kinematic parameters, especially on the second order, will remain more constant between domains.

REFERENCES

[1] H. Cui, T. Nguyen, F.-C. Chou, T.-H. Lin, J. Schneider, D. Bradley, and N. Djuric, “Deep kinematic models for kinematically feasible vehicle trajectory predictions,” in 2020 IEEE International Conference on Robotics and Automation (ICRA), p. 10563–10569, IEEE, May 2020.

[2] C. Anderson, R. Vasudevan, and M. Johnson-Roberson, “A kinematic model for trajectory prediction in general highway scenarios,” IEEE Robotics and Automation Letters, vol. 6, p. 6757–6764, Oct 2021.

[3] S. Suo, K. Wong, J. Xu, J. Tu, A. Cui, S. Casas, and R. Urtasun, “Mixsim: A hierarchical framework for mixed reality traffic simulation,” in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 9622–9631, IEEE, Jun 2023.

[4] B. Varadarajan, A. Hefny, A. Srivastava, K. S. Refaat, N. Nayakanti, A. Cornman, K. Chen, B. Douillard, C. P. Lam, D. Anguelov, et al., “Multipath++: Efficient information fusion and trajectory aggregation for behavior prediction,” in 2022 International Conference on Robotics and Automation (ICRA), pp. 7814–7821, IEEE, 2022.

[5] F. Farahi and H. S. Yazdi, “Probabilistic kalman filter for moving object tracking,” Signal Processing: Image Communication, vol. 82, p. 115751, 2020.

[6] C. G. Prevost, A. Desbiens, and E. Gagnon, “Extended kalman filter for state estimation and trajectory prediction of a moving object detected by an unmanned aerial vehicle,” in 2007 American Control Conference, p. 1805–1810, IEEE, July 2007.

[7] S. Shi, L. Jiang, D. Dai, and B. Schiele, “Motion transformer with global intention localization and local movement refinement,” Advances in Neural Information Processing Systems, 2022.

[8] S. Ettinger, S. Cheng, B. Caine, C. Liu, H. Zhao, S. Pradhan, Y. Chai, B. Sapp, C. R. Qi, Y. Zhou, Z. Yang, A. Chouard, P. Sun, J. Ngiam, V. Vasudevan, A. McCauley, J. Shlens, and D. Anguelov, “Large scale interactive motion forecasting for autonomous driving: The waymo open motion dataset,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9710–9719, October 2021.

[9] K. Chen, R. Ge, H. Qiu, R. Ai-Rfou, C. R. Qi, X. Zhou, Z. Yang, S. Ettinger, P. Sun, Z. Leng, M. Mustafa, I. Bogun, W. Wang, M. Tan, and D. Anguelov, “Womd-lidar: Raw sensor dataset benchmark for motion forecasting,” arXiv preprint arXiv:2304.03834, April 2023.

[10] B. Wilson, W. Qi, T. Agarwal, J. Lambert, J. Singh, S. Khandelwal, B. Pan, R. Kumar, A. Hartnett, J. Kaesemodel Pontes, D. Ramanan, P. Carr, and J. Hays, “Argoverse 2: Next generation datasets for selfdriving perception and forecasting,” in Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks (J. Vanschoren and S. Yeung, eds.), vol. 1, Curran, 2021.

[11] H. Caesar, V. Bankiti, A. H. Lang, S. Vora, V. E. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, and O. Beijbom, “nuscenes: A multimodal dataset for autonomous driving,” in CVPR, 2020.

[12] C. Qian, D. Xiu, and M. Tian, “The 2nd place solution for 2023 waymo open sim agents challenge,” 2023.

[13] Y. Liu, J. Zhang, L. Fang, Q. Jiang, and B. Zhou, “Multimodal motion prediction with stacked transformers,” Computer Vision and Pattern Recognition, 2021.

[14] Z. Zhou, J. Wang, Y.-H. Li, and Y.-K. Huang, “Query-centric trajectory prediction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.

[15] J. Ngiam, V. Vasudevan, B. Caine, Z. Zhang, H.-T. L. Chiang, J. Ling, R. Roelofs, A. Bewley, C. Liu, A. Venugopal, et al., “Scene transformer: A unified architecture for predicting future trajectories of multiple agents,” in International Conference on Learning Representations, 2021.

[16] Y. Chai, B. Sapp, M. Bansal, and D. Anguelov, “Multipath: Multiple probabilistic anchor trajectory hypotheses for behavior prediction,” in Proceedings of the Conference on Robot Learning (L. P. Kaelbling, D. Kragic, and K. Sugiura, eds.), vol. 100 of Proceedings of Machine Learning Research, pp. 86–99, PMLR, 30 Oct–01 Nov 2020.

[17] A. Scibior, V. Lioutas, D. Reda, P. Bateni, and F. Wood, “Imagining ´ the road ahead: Multi-agent trajectory prediction via differentiable simulation,” in 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), p. 720–725, IEEE, Sep 2021.

[18] D. Rempe, J. Philion, L. J. Guibas, S. Fidler, and O. Litany, “Generating useful accident-prone driving scenarios via a learned traffic prior,” in Conference on Computer Vision and Pattern Recognition (CVPR), 2022.

[19] M. Lutter, C. Ritter, and J. Peters, “Deep lagrangian networks: Using physics as model prior for deep learning,” in International Conference on Learning Representations (ICLR), 2019.

[20] S. Greydanus, M. Dzamba, and J. Yosinski, “Hamiltonian neural networks,” in Advances in Neural Information Processing Systems (H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alche-Buc, E. Fox, ´ and R. Garnett, eds.), vol. 32, Curran Associates, Inc., 2019.

[21] M. Janner, J. Fu, M. Zhang, and S. Levine, “When to trust your model: Model-based policy optimization,” in Advances in Neural Information Processing Systems, 2019.

[22] F. de Avila Belbute-Peres, K. Smith, K. Allen, J. Tenenbaum, and J. Z. Kolter, “End-to-end differentiable physics for learning and control,” Advances in neural information processing systems, vol. 31, 2018.

[23] J. Degrave, M. Hermans, J. Dambre, et al., “A differentiable physics engine for deep learning in robotics,” Frontiers in neurorobotics, p. 6, 2019.

[24] M. Geilinger, D. Hahn, J. Zehnder, M. Bacher, B. Thomaszewski, ¨ and S. Coros, “Add: Analytically differentiable dynamics for multibody systems with frictional contact,” ACM Transactions on Graphics (TOG), vol. 39, no. 6, pp. 1–15, 2020.

[25] Y.-L. Qiao, J. Liang, V. Koltun, and M. C. Lin, “Scalable differentiable physics for learning and control,” in ICML, 2020.

[26] S. Son, L. Zheng, R. Sullivan, Y. Qiao, and M. Lin, “Gradient informed proximal policy optimization,” in Advances in Neural Information Processing Systems, 2023.

[27] J. Xu, V. Makoviychuk, Y. Narang, F. Ramos, W. Matusik, A. Garg, and M. Macklin, “Accelerated policy learning with parallel differentiable simulation,” in International Conference on Learning Representations, 2021.

[28] J. Liang, M. Lin, and V. Koltun, “Differentiable cloth simulation for inverse problems,” Advances in Neural Information Processing Systems, vol. 32, 2019.

[29] Y. Chai, B. Sapp, M. Bansal, and D. Anguelov, “Multipath: Multiple probabilistic anchor trajectory hypotheses for behavior prediction,” in Proceedings of the Conference on Robot Learning (L. P. Kaelbling, D. Kragic, and K. Sugiura, eds.), vol. 100 of Proceedings of Machine Learning Research, pp. 86–99, PMLR, 30 Oct–01 Nov 2020.

[30] B. Varadarajan, A. Hefny, A. Srivastava, K. S. Refaat, N. Nayakanti, A. Cornman, K. Chen, B. Douillard, C. P. Lam, D. Anguelov, and B. Sapp, “Multipath++: Efficient information fusion and trajectory aggregation for behavior prediction,” in 2022 International Conference on Robotics and Automation (ICRA), p. 7814–7821, IEEE Press, 2022.

[31] Y. Wang, T. Zhao, and F. Yi, “Multiverse transformer: 1st place solution for waymo open sim agents challenge 2023,” arXiv preprint arXiv:2306.11868, 2023.

[32] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” 2022.

[33] B. M. Albaba and Y. Yildiz, “Driver modeling through deep reinforcement learning and behavioral game theory,” IEEE Transactions on Control Systems Technology, vol. 30, no. 2, pp. 885–892, 2022.