Related papers: DEA-based benchmarking for performance evaluation …
Many definitions of business processes refer to business goals, value creation, or profits/gains of sorts. Nevertheless, the focus of formal methods research on business processes, like the well-known soundness property, lies on correctness…
Software engineering considers performance evaluation to be one of the key portions of software quality assurance. Unfortunately, there seems to be a lack of standard methodologies for performance evaluation even in the scope of…
Evaluation of students' performance for the completion of courses has been a major problem for both students and faculties during the work-from-home period in this COVID pandemic situation. To this end, this paper presents an in-depth…
In data envelopment analysis (DEA) literature, the returns to scale (RTS) of an inefficient decision making unit (DMU) is determined at its projected point on the efficient frontier. Under the occurrences of multiple projection points,…
We consider Incentive Decision Processes, where a principal seeks to reduce its costs due to another agent's behavior, by offering incentives to the agent for alternate behavior. We focus on the case where a principal interacts with a…
Ranking entities such as algorithms, devices, methods, or models based on their performances, while accounting for application-specific preferences, is a challenge. To address this challenge, we establish the foundations of a universal…
Data-centric learning seeks to improve model performance from the perspective of data quality, and has been drawing increasing attention in the machine learning community. Among its key tools, influence functions provide a powerful…
In multi-agent reinforcement learning systems, the actions of one agent can have a negative impact on the rewards of other agents. One way to combat this problem is to let agents trade their rewards amongst each other. Motivated by this,…
Models that top leaderboards often perform unsatisfactorily when deployed in real world applications; this has necessitated rigorous and expensive pre-deployment model testing. A hitherto unexplored facet of model performance is: Are our…
In recent years, various decentralized organizational forms have emerged, posing a challenge for organizational design. Some design elements, such as task allocation, become emergent properties that cannot be fully controlled from the top…
Identification of the reference set for each decision making unit (DMU) is a main concern in the data envelopment analysis (DEA). All of the methods developed to date have been focused on finding the furthest reference DMUs. In this paper,…
The M6 Competition assessed the performance of competitors using a ranked probability score and an information ratio (IR). While these metrics do well at picking the winners in the competition, crucial questions remain for investors with…
It is an important management task of metro agencies to formulate reasonable improvement schemes based on the result of service quality surveys. Considering scores, weights, and improvement feasibility of service quality attributes in a…
Critical sectors of human society are progressing toward the adoption of powerful artificial intelligence (AI) agents, which are trained individually on behalf of self-interested principals but deployed in a shared environment. Short of…
Promotions have been trending in the e-commerce marketplace to build up customer relationships and guide customers towards the desired actions. Since incentives are effective to engage customers and customers have different preferences for…
Incentives play an important role in (security and IT) risk management of a large-scale organization with multiple autonomous divisions. This paper presents an incentive mechanism design framework for risk management based on a…
We consider the design of experiments to evaluate treatments that are administered by self-interested agents, each seeking to achieve the highest evaluation and win the experiment. For example, in an advertising experiment, a company wishes…
Research and development (R&D) of countries play a major role in a long-term development of the economy. We measure the R&D efficiency of all 28 member countries of the European Union in the years 2008--2014. Super-efficient data…
Reward models (RMs) play a crucial role in reinforcement learning from human feedback (RLHF), aligning model behavior with human preferences. However, existing benchmarks for reward models show a weak correlation with the performance of…
Reward models are used throughout the post-training of language models to capture nuanced signals from preference data and provide a training target for optimization across instruction following, reasoning, safety, and more domains. The…