Assalam O Alaikum Wa Rahmatullahi Wa Barakatuh,
Most likely you are refering to hadith in Sunan Abi Dawud. Hadith is quoted below with reference:
Narrated Abu Hurairah:
A man was praying with his lower garment hanging down. The Messenger of Allah (ﷺ) said to him: Go and perform ablution. He then went and performed ablution. He then came and he said: Go and perform ablution. Then a man said to him: Messenger of Allah, what is the matter with you that you commanded him to perform ablution and then you kept silence ?
He replied: He was praying while hanging down his lower garments, and Allah does not accept the prayer of a man who hangs down his lower garment.
حَدَّثَنَا مُوسَى بْنُ إِسْمَاعِيلَ، حَدَّثَنَا أَبَانُ، حَدَّثَنَا يَحْيَى، عَنْ أَبِي جَعْفَرٍ، عَنْ عَطَاءِ بْنِ يَسَارٍ، عَنْ أَبِي هُرَيْرَةَ، قَالَ بَيْنَمَا رَجُلٌ يُصَلِّي مُسْبِلاً إِزَارَهُ فَقَالَ لَهُ رَسُولُ اللَّهِ صلى الله عليه وسلم " اذْهَبْ فَتَوَضَّأْ " . فَذَهَبَ فَتَوَضَّأَ ثُمَّ جَاءَ ثُمَّ قَالَ " اذْهَبْ فَتَوَضَّأْ " . فَقَالَ لَهُ رَجُلٌ يَا رَسُولَ اللَّهِ مَا لَكَ أَمَرْتَهُ أَنْ يَتَوَضَّأَ ثُمَّ سَكَتَّ عَنْهُ قَالَ " إِنَّهُ كَانَ يُصَلِّي وَهُوَ مُسْبِلٌ إِزَارَهُ وَإِنَّ اللَّهَ لاَ يَقْبَلُ صَلاَةَ رَجُلٍ مُسْبِلٍ " .
Grade: Da'if (Al-Albani)
Reference: Sunan Abi Dawud 4086
In-book reference: Book 34, Hadith 67
English translation: Book 33, Hadith 4075
http://sunnah.com/abudawud/34/67
Now this hadith is Daif as you pointed out but there are many others hadiths on similar topic (lowering garments below ankles) which are Sahih. For example:
Narrated Abdur Rahman:
I asked Abu Sa'id al-Khudri about wearing lower garment. He said: You have come to the man who knows it very well. The Messenger of Allah (ﷺ) said: The way for a believer to wear a lower garment is to have it halfway down his legs and he is guilty of no sin if it comes halfway between that and the ankles, but
what comes lower than the ankles is in Hell. On the day of Resurrection. Allah will not look at him who trails his lower garment conceitedly.
حَدَّثَنَا حَفْصُ بْنُ عُمَرَ، حَدَّثَنَا شُعْبَةُ، عَنِ الْعَلاَءِ بْنِ عَبْدِ الرَّحْمَنِ، عَنْ أَبِيهِ، قَالَ سَأَلْتُ أَبَا سَعِيدٍ الْخُدْرِيَّ عَنِ الإِزَارِ، فَقَالَ عَلَى الْخَبِيرِ سَقَطْتَ قَالَ رَسُولُ اللَّهِ صلى الله عليه وسلم " إِزْرَةُ الْمُسْلِمِ إِلَى نِصْفِ السَّاقِ وَلاَ حَرَجَ - أَوْ لاَ جُنَاحَ - فِيمَا بَيْنَهُ وَبَيْنَ الْكَعْبَيْنِ مَا كَانَ أَسْفَلَ مِنَ الْكَعْبَيْنِ فَهُوَ فِي النَّارِ مَنْ جَرَّ إِزَارَهُ بَطَرًا لَمْ يَنْظُرِ اللَّهُ إِلَيْهِ " .
Grade: Sahih (Al-Albani)
Reference: Sunan Abi Dawud 4093
In-book reference: Book 34, Hadith 74
English translation: Book 33, Hadith 4082
https://sunnah.com/abudawud/34/74
It is narrated on the authority of Abu Dharr who narrates that the Prophet (ﷺ) observed:
Three are the persons with whom
Allah would not speak on the Day of Resurrection: the bestower of gift who does not give anything but by laying obligation on him, the seller of goods who sells them by taking false oath and one
who hangs low his lower garment.
وَحَدَّثَنِي أَبُو بَكْرِ بْنُ خَلاَّدٍ الْبَاهِلِيُّ، حَدَّثَنَا يَحْيَى، - وَهُوَ الْقَطَّانُ - حَدَّثَنَا سُفْيَانُ، حَدَّثَنَا سُلَيْمَانُ الأَعْمَشُ، عَنْ سُلَيْمَانَ بْنِ مُسْهِرٍ، عَنْ خَرَشَةَ بْنِ الْحُرِّ، عَنْ أَبِي ذَرٍّ، عَنِ النَّبِيِّ صلى الله عليه وسلم قَالَ " ثَلاَثَةٌ لاَ يُكَلِّمُهُمُ اللَّهُ يَوْمَ الْقِيَامَةِ الْمَنَّانُ الَّذِي لاَ يُعْطِي شَيْئًا إِلاَّ مَنَّهُ وَالْمُنَفِّقُ سِلْعَتَهُ بِالْحَلِفِ الْفَاجِرِ وَالْمُسْبِلُ إِزَارَهُ " .
Reference: Sahih Muslim 106 b
In-book reference: Book 1, Hadith 200
USC-MSA web (English) reference: Book 1, Hadith 193
https://sunnah.com/muslim/1/200
Scholars have difference of opinion on this issue. Some say that lowering garments below ankles in a major sin and warnings in hadith should be taken seriously. Others say that lowering garments above ankle was to get rid of pride as it was custom of Arabs at that time to let their garments slide on the ground to show their pride. Now a days, even a poor person can afford to have garments which will go below ankles. So basically, they say that if you lower your garments without having any pride then its not a sin. Below are some links from scholars who supports both the opinions I mentioned above.
https://islamqa.info/en/answers/762/ruling-on-wearing-one146s-clothes-below-one146s-ankles
https://seekersguidance.org/answers/general-counsel/trousers-below-ones-ankles/
You can read both opinions to educate yourself and may be it is best to ask any questions to an Imam in your local community. If you are unable to decide which way to go then follow whatever your local Imam will say.
And Allah knows the best!
Ma'a Salama