One of the most challenging factors in the development of autonomous vehicles and advanced driver assistance systems is the imitation of an expert driver system which is the observer and interpreter of the technical system in the related driving scenario. In this paper, a multimodal adaptive driver assistance system is presented. The main goal is to determine the human driver's attention and authority level by decoupling the driver's vehicle control in the longitudinal and lateral direction in order to trigger timely warnings according to his/her driving intents and driving skills with respect to the possible driving situation and hazard scenarios. The presented driver assistance system considers the driver's driving performance metric sampled during the longitudinal and lateral vehicle control tasks as well as the processed information about the surrounding traffic environment consisting of the interactions with the other vehicles and the road situations. Experiments on a simulator are performed and the presented metric is calculated for the evaluation of the human driver's driving performance with respect to adaptive cruising and obstacle avoidance maneuvering tasks.