You can permanently delete a SharePoint Online Office 365 Group from your Azure Portal. But if you prefer to use PowerShell, here is an example.
Install-Module -Name AzureAD
$tenantUrl = "https://yoursite-admin.sharepoint.com/"
$groupToDelete = "FACT"
Connect-SPOService -Url $tenantUrl
Write-Output "Connected to $tenantUrl"
Get-AzureADMSDeletedGroup $groupToDelete | Write-Output "Group $groupToDelete was found"
#Remove-AzureADMSDeletedDirectoryObject –Id $groupToDelete