@Fredo use the HistoryService to create task query. You are going to query for all tasks for your process instance. Then get the assignee of the task for each task. You can then manipulate your data to have a Map of [taskId : assignee] which you can then do your checks on who did which tasks.
For implementing actual restrictions, you could implement User Task Event Completion listener that checks whats going on and if being completed by the wrong user then throw a error which will trigger a rollback to before the task was submitted.
You could also implement a logic on the user task to determine if the User Task can be assigned/and thus claimed by a user. This would be a listener on the Assigned event. You would be checking against your Map and throw a exception if it fails to meet your logic.
Everything Stephen said, but: querying the history service for runtime decisions can easily become a performance trap, since the historic data is fast growing. It may be worth considering an extra process variable for this kind of runtime data.