Polarization encodes vectorial optical information invisible to conventional intensity-only sensors but critical for advanced artificial vision. Implementing in-sensor computing along the polarization dimension, however, remains challenging due to the lack of reconfigurable array-level hardware. Here, we demonstrate self-reconfigurable polarization sensor arrays built from epitaxial van der Waals heterostructures of 2D tellurium and ReS₂ with orthogonal in-plane anisotropies. Light-driven interfacial carrier transfer and trapping synergistically produce a bipolar photoresponse, while perpendicular lattice alignment yields opposite polarization sensitivities in the two response branches, enabling all-optical reconfiguration without electrical bias. Integrated into in-sensor architectures, these arrays process time-resolved, multi-dimensional optical information with high fidelity. Dynamic vision tasks achieve enhanced accuracy (>95%), demonstrated in applications ranging from autonomous driving to intelligent medicine. Our results establish self-reconfigurable polarization arrays as a versatile platform for high-dimensional in-sensor photonic computing, offering a path toward efficient artificial vision in complex real-world environments.