Querying distinct values from NoSQL(Cosmos DB specifically) cannot convert to model type - c#

Summary
In short I am trying to display all the distinct Ids. I am trying to understand why when I first ran my query (not being distinct) I was able to use them with my model as opposed to now. The error that I am getting is Cannot convert from System.String to AppName.Models.Student.
Example
For example displaying all ids from my collection (with repeated ids in documents) : "SELECT s.ClassId from Students s" worked fine, but when I tried to get the distinct values it couldn't convert them, with the query being SELECT DISTINCT VALUE s.ClassId from Students s"
This is my query to Cosmos DB:
public List<T> Query(string query)
{
FeedOptions queryOptions = new FeedOptions { MaxItemCount = -1, EnableCrossPartitionQuery = true };
// Execute query via SQL
IQueryable<T> QueryInSql = Client.CreateDocumentQuery<T>(
UriFactory.CreateDocumentCollectionUri(Database, Collection),
$"{query}",
queryOptions);
// convert Iqueryable to a list
return QueryInSql.ToList();
}
I changed my model to just include
public class Class
{
public string ClassId {get; set;}
}
Sample collection with documents
{
{
"id":"abc",
"studentName": "bob",
"classId":"en"
},
{
"id":"bcd",
"studentName": "billy",
"classId":"sp"},
{
"id":"sdf",
"studentName": "bart",
"classId":"en"
}
}

So, the error was in the query I passed in. When adding the VALUE to it, you get back
["en","sp"]
However what I thought I as getting was
[{"id": "en"}, {"id": "sp"}]

Related

Find in list of object all items with property exists in other list

i'm new here. My problem is this:
List<string> positions = new List<string> { "x1", "x2", "x3 };
var results = _context_WObjects.AsQueryAble();
Result returns list of IQueryAble list that have above class:
class objectclass {
public string position {get; set; }
public string test {get; set; }
/*... other properties....*/
}
How can i find all items with position that has (not contains) one of positions's list?
I've tried:
results = results.Where(n1 => positions.Any(n2 => n2 == n1.position));
But it wouldn't working!!
The exception error is:
"Local sequence can not be used in LINQ to SQL implementations of query operators except for the Contains operator."
I've translated from italian.
Thanks
Very much
Cris
The error message is pretty clear. You cannot use Any with a local collection like this List<string> but you can use Contains. You also want !Contains:
results = results.Where(n1 => !positions.Contains(n1.position));

DocumentDB unsupported query

I try to use dictionary in IQueryable but I received run time error ,I know the problem occur because in real time IQueryable is not familiar with that object , I try to convert IQueryable to IEnumerable , but I have problem with the execution of the query.
May someone can give me a hint how to execute that function ?
I have the following code:
Dictionary<String, int> coursesType= new Dictionary<string, int>();
var userQuery = _client.CreateDocumentQuery<ObjectModel.Student>(uriStudentCollection, options).
Where(x =>coursesType.ContainsKey(x.MainCourse)
&& !x.Courses.ContainsKey(requestCourse)).OrderBy(x => x.Grade).AsDocumentQuery();
var feedResponse = await userQuery.ExecuteNextAsync<ObjectModel.Student>();
foreach (var ad in feedResponse.AsEnumerable())
{
results.Add(ad);
}
UPDATE STATUS: I STILL NOT RECEIVED ANSWER TO MY QUESTION
***UPDATE : I add example of my doc.
{
"id": "a5d7f123-80d5-5094-84fb-08c3bc4ccp972",
"StudentName": "Philip",
"Courses": {
"Math": {
"id": "Math",
"Grade": "98",
"Place": "NYC"
}
},
"Rank":"AA"
}
UPDATE NUMBER 3
I write the following query :
SqlQuerySpec q = new SqlQuerySpec()
{
QueryText = "SELECT * FROM c WHERE (CONTAINS(LOWER(c[\"courseName\"]),#text) OR CONTAINS(LOWER(c[\"courseDescription\"]),#text) ) AND (udf.CourseContainsKey(c[\"Courses\"],#courseId)=false)",
Parameters = new SqlParameterCollection()
{
new SqlParameter("#text", text),
new SqlParameter("#courseId", courseId)
}
};
When I write the query like that, the query work fine, but IF I add the ORDER BY command to the query I received empty set....
"SELECT * FROM c WHERE (CONTAINS(LOWER(c[\"courseName\"]),#text) OR CONTAINS(LOWER(c[\"courseDescription\"]),#text) ) AND (udf.CourseContainsKey(c[\"Courses\"],#courseId)=false) ORDER BY c[\"courseName\"] ASC"
Thanks
Thanks,
MAK
{"Method 'ContainsKey' is not supported."}
Based on your query, you could use the following code:
var userQuery = _client.CreateDocumentQuery<ObjectModel.Student>(uriStudentCollection, options).
Where(x =>coursesType.Keys.Contains(x.MainCourse)
&& !x.Courses.Keys.Contains(requestCourse)).OrderBy(x => x.Grade).AsDocumentQuery();
Additionally, if you enable Cross partition query, you would get the following error:
Cross partition query with TOP/ORDER BY or aggregate functions is not supported.
If the filter could not executed on CosmosDB side, I assumed that you need to pull the records from azure side, then filter on your client side. Additionally, here is a similar issue, you could refer to here.
UPDATE:
Sample document:
{
"id": "1ba6178b-7c22-440a-a4a2-25b4bc636b30",
"MainCourse": "b",
"Grade": "B",
"Courses": {
"a": "a",
"b": "b"
}
}
Query:
SELECT * FROM root WHERE ((root["MainCourse"] IN ("a", "b")) AND (root["Courses"]["a"] != null)) ORDER BY root["Grade"] ASC
Modify your C# code as follows:
!x.Courses.Keys.Contains(requestCourse)
//To
x.Courses[requestCourse]==null
UPDATE2:
For filtering a specific course name not contains within the Courses property, I assumed that you could use User-defined functions, here is the code snippet, you could refer to it:
udf.CourseContainsKey:
function CourseContainsKey (courses,courseName) {
if(courses==undefined||courseName==undefined)
return false;
return courses.hasOwnProperty(courseName);
}
Test:
UPDATE3:
I have not found any better way to create the query with the UDF, you could follow the code below to create your document query:
var query = $"SELECT* FROM root WHERE (root[\"MainCourse\"] IN({String.Join(",", coursesType.Keys.Select(k=>$"\"{k}\"").ToArray())})) AND udf.CourseContainsKey(root[\"Courses\"],\"{requestCourse}\")=false ORDER BY root[\"Grade\"] ASC";
var items=client.CreateDocumentQuery<CourseSample>(UriFactory.CreateDocumentCollectionUri(DatabaseId, DocumentCollectionId), sqlExpression:query).ToList();

Dapper.net “where … in” query doesn't work with PostgreSQL

The following query always produces the error "42601: syntax error at or near "$1"
".
connection.Query<CarStatsProjection>(
#"select manufacturer, model, year, AVG(price) as averageprice, AVG(miles) as averagemiles, COUNT(*) as count
from products
where manufacturer IN #manufacturers
AND model IN #models
AND year IN #years
group by manufacturer, model, year",
new { manufacturers = new[] { "BMW", "AUDI" },
models = new[] { "M4", "A3" },
years = new[] { 2016, 2015 } });
I have got around this by creating a method below and calling it inline to build the SQL query for now. Would like to know if Dapper can handle this with the object param though?
public static string ToInSql(this IEnumerable<object> values)
{
var flattened = values.Select(x => $"'{x}'");
var flatString = string.Join(", ", flattened);
return $"({flatString})";
}
PostgreSQL IN operator doesn't support array (or any other collection) as parameter, only a normal list (the one which you're generating with the ToInSql method), for PostgreSQL you need to use ANY operator, like this:
SELECT manufacturer, model, year, AVG(price) as averageprice, AVG(miles) as averagemiles, COUNT(*) as count
FROM products
WHERE manufacturer = ANY(#manufacturers)
AND model = ANY(#models)
AND year = ANY(#years)
GROUP BY manufacturer, model, year

Filtering selected JSon fields in the DB side

This is my DAL method:
public List<Schedule> GetSchedulesWithProfiles(int displayStart, int displayLength, out int allDataCount, out int filteredDatacount, string searchParam = "", string searchDir = "")
{
using (var context = new ApplicationDbContext())
{
var schedules = context.Schedules.Include(x => x.Profile).Include(x => x.VacationType);
allDataCount = schedules.Count();
if (!string.IsNullOrEmpty(searchParam))
{
schedules = schedules.Where(c => c.Data.Contains(searchParam));
}
filteredDatacount = schedules.Count();
if (searchDir == "asc" || String.IsNullOrEmpty(searchDir))
schedules = schedules.OrderBy(x => x.Data).Skip(displayStart).Take(displayLength);
else
schedules = schedules.OrderByDescending(x => x.Data).Skip(displayStart).Take(displayLength);
return schedules.ToList();
}
}
As you can see, I do filtering by Data fieled in my Db. But in the DB I have Data in this format:
{
"FirstName": "Alex",
"LastName": "Alex",
"Position": "dev"
}
This is just nvarchar in the DB. I need to do filtering by FirstName and/or LastName. How I can implement it? Is exist any way to Deserialize Object in IQueryable LINQ in the DB side?
Currently there is no native support for JSON data in SQL server: feature request
Also you cannot deserialize string on database side. You need to get string from database, then deserialize it locally.
I would suggest you to create three columns FirstName, LastName and Position instead of storing serialized data. Or if you want data to be flexible think about using appropriate database (e.g. MongoDB)
Worth reading: Consuming JSON Strings in SQL Server. As Phil says, parsing JSON is something inappropriate in SQL. But you can use his SQL function to convert your JSON data into table format.
Side notes:
To get count of all schedules use allDataCount = context.Schedules.Count(). You don't need to include related entities into generated query if you just looking for count.
This line filteredDatacount = schedules.Count() executes one more query, which you don't need. Simply get count after you got all filtered entities:
var filteredSchedules = schedules.ToList();
filteredDatacount = filteredSchedules.Count;
return filteredSchedules;

Tracking down a stack overflow error in my LINQ query

I've written the following LINQ query:
IQueryable<ISOCountry> entries =
(from e in competitorRepository.Competitors
join c in countries on e.countryID equals c.isoCountryCode
where !e.Deleted
orderby c.isoCountryCode
select new ISOCountry() { isoCountryCode = e.countryID, Name = c.Name }
).Distinct();
The objective is to retrieve a list of the countries represented by the competitors found in the system. 'countries' is an array of ISOCountry objects explicitly created and returned as an IQueryable<ISOCountry> (ISOCountry is an object of just two strings, isoCountryCode and Name). Competitors is an IQueryable<Competitor> which is bound to a database table through LINQ to SQL though I created the objects from scratch and used the LINQ data mapping decorators.
For some reason, this query causes a stack overflow when the system tries to execute it. I've no idea why, I've tried trimming the Distinct, returning an anonymous type of the two strings, using 'select c', but all result in the overflow. The e.CountryID value is populated from a dropdown that was in itself populated from the IQueryable<ISOCountry>, so I know the values are appropriate but even if not, I wouldn't expect a stack overflow.
Why is the overflow is occurring or why might it be happening?
As requested, code for ISOCountry:
public class ISOCountry
{
public string isoCountryCode { get; set; }
public string Name { get; set; }
}
It's initialised from a static utility class thus:
public static IQueryable<ISOCountry> GetCountryCodes()
{
// ISO 3166-1 country names and codes from http://opencountrycodes.appspot.com/javascript
ISOCountry[] countries = new ISOCountry[] {
new ISOCountry { isoCountryCode= "AF", Name= "Afghanistan"},
new ISOCountry { isoCountryCode= "AX", Name= "Aland Islands"},
new ISOCountry { isoCountryCode= "AL", Name= "Albania"},
new ISOCountry { isoCountryCode= "DZ", Name= "Algeria"},
new ISOCountry { isoCountryCode= "AS", Name= "American Samoa"},
...
new ISOCountry { isoCountryCode= "YE", Name= "Yemen"},
new ISOCountry { isoCountryCode= "ZM", Name= "Zambia"},
new ISOCountry { isoCountryCode = "ZW", Name = "Zimbabwe"}
};
return countries.AsQueryable();
}
How I finally got it to work, see below... I am still curious as to what specifically is wrong with the original query, I'm sure I've done similar things before.
IList<string> entries = competitorRepository.Competitors.Select(c=>c.CountryID).Distinct().ToList();
IList<ISOCountry> countries = Address.GetCountryCodes().Where(a => entries.Contains(a.isoCountryCode)).ToList();
Maybe I'm crazy, but your utility class shouldn't be outputting an IQueryable list. You're creating a local sequence that looks like it should be queryable. Ultimately, IQueryable lists should be delved out by your datacontext. If a utility class is creating a list, that should be returned as (most likely) an array or an IEnumerable, for example:
public static readonly ISOCountry[] CountryCodes = new ISOCountry[] {
new ISOCountry { isoCountryCode= "AF", Name= "Afghanistan"},
new ISOCountry { isoCountryCode= "AX", Name= "Aland Islands"}
...
};
A local sequence can only be used in an IQueryable .Contains() statement. So, if you want to "mesh" your local sequence with your IQueryable sequence, you have to force the IQueryable to fire a SQL statement and grab the records it represents from the database. To do that, all you have to do is iterate over the IQueryable records in some fashion:
IList<Competitor> competitorRecords = competitorRepository
.Competitors
.Where(m => !m.Deleted)
.OrderBy(m => m.countryId)
.ToList(); //This fires the SQL statement
Once you've snagged the records from the database, you can create your list of ISOCountry records. Again, since this list isn't coming from your datacontext, it shouldn't be an IQueryable list. Instead, try this:
IList<ISOCountry> = competitorRecords
.Join(CountryCodes, key1 => key1.countryId, key2 => key2.isoCountryCode, (competitors, codes) => new ISOCountry { isoCountryCode = competitors.countryId, Name = codes.Name })
.ToList();
This will work, but you're probably grabbing unnecessary records from the database. It'd be even better if you could upload your ISOCountry list to the database. Once you do that, you'd be able to fire the query as you initially conceived it.

Resources