OBJECTIVES: In the context of uncertainty due to the lack of sound data, expert opinion is considered as a legitimate source of information for decision-makers. The use of experts’ opinion requires to quantifying their uncertainty about a specific event by eliciting a probability distribution of the event. The objectives of this study were to develop a web-based tool enabling users to remotely elicit the opinion of a group of geographically dispersed experts and to evaluate the measurement properties of this tool. METHODS: The web-based tool allowed first to elicit univariate probability distributions separately from each expert and secondly to calculate an aggregated distribution. The elicitation method was the four-interval method that was judged to be more appropriate for non-statistician experts due to its clarity of use. As recommended to limit biases, the elicitation questionnaire included a training exercise and a graphical feedback so that the experts could validate their distributions. A pilot survey was conducted among all the French regional medical device vigilance correspondents (n=24) about the risk of failure (%) of an implantable medical device.  RESULTS: Twenty-two correspondents (92%) completed the survey. An aggregated distribution was calculated from the elicited individual distributions and a beta distribution was fitted reflecting the group uncertainty about the risk of failure. Feasibility was judged in view of the users’ feedback and time to completion. Validity and reliability were assessed using data on comprehensiveness, internal coherence and test-retest reliability.  CONCLUSIONS: The proposed web-based tool was feasible, valid and reliable. It should be useful in making expert elicitation easier and more practical.