Hungary is bringing in tough new laws to stop the entry of illegal refugees.
Hungarian police can now detain anyone who tries to breach a razor-wire fence built on the border with Serbia.
The country has become a key point on the journey north for thousands of refugees from the Middle East and Africa.
The EU has agreed to relocate 40,000 refugees from Greece and Italy to other EU states, starting on September 15.
However, it failed to agree on mandatory quotas for a further 120,000 asylum seekers.
Instead, at the meeting in Brussels, a majority of ministers agreed “in principle” and negotiations will now take place ahead of another meeting in October.
The new Hungarian laws came into effect at on September 14 at midnight.
Police sealed a railway crossing point that had been used by tens of thousands of refugees, and many slept out in the open on the Serbian side of the border.
Police buses will now take asylum applicants to registration centers, but if their applications are refused they will now be returned to Serbia rather than being given passage through Hungary.
Hungarian authorities said more than 9,000 – a new record – crossed into the country before the border was closed on September 14.
From now on anyone who crosses the border illegally will face criminal charges, and 30 judges have been put on standby to try potential offenders.
Mounted police have been deployed along the border.
“We will start a new era,” government spokesman Zoltan Kovacs said shortly after midnight.
“We will stop the inflow of illegal migrants over our green borders.”
Zoltan Kovacs added: “That also means that the official and legal ways to come to Hungary and therefore to the European Union remain open. That’s all we ask from all migrants – that they should comply with international and European law.”
At the Brussels talks, Luxembourg, which holds the EU presidency, said it was hoped that the relocation proposal could be made law at a meeting on October 8.
Hungary, the Czech Republic and Slovakia were reportedly among the nations opposed to mandatory quotas.
“There was no consensus, several countries disagreed,” Slovak Interior Minister Robert Kalinak said after the talks.
Luxembourg Foreign Minister Jean Asselborn said “not everyone is on board at the moment”.
He warned the situation in Europe was “urgent and dramatic and time is of the essence”.
Under complex EU rules, a unanimous vote is not required and decisions can be made with a qualified majority.
However, correspondents say that would be a show of disunity that the EU is trying to avoid.
Jean Asselborn said a list of safe countries, to which failed asylum seekers can be returned, had been agreed on principle.
European states have been struggling to cope with a record influx of refugees, who are mainly trying to reach Germany and Sweden.
On September 14, a number of EU countries followed Germany’s suit in introducing temporary border checks.
The moves are a challenge to the EU’s Schengen agreement on free movement, although the rules do allow for temporary controls in emergencies.
Austrian Chancellor Werner Faymann said troops were also being deployed, mainly to provide humanitarian help within Austria, but would be sent to the border if necessary.