Restricting Peer-Level Access in Employee Hierarchy

We have a requirement regarding access to Employee-related documents. The goal is to allow certain users to view all Employee-related data within their department or associated department without setting individual user permissions, granting them access to all Employee records.

However, they should not be able to view data for their peers or employees higher up in the hierarchy. While access to higher-level employees can be managed using the left and right (lft and rgt) values in the Employee tree structure, restricting access to peer data remains a challenge.

Has anyone faced a similar scenario or has suggestions on how to approach this effectively?