When encoding HTML input for security, how do I avoid encoding international characters like Ñ or ñ?

I have a textarea in an ASP.NET MVC Application where the user can type some text. When I show the text to the user, I use Html.Encode to prevent malicious input. The problem is that the user can type in Spanish and maybe he types año and the Encode transforms this into a&#241o. How Can I prevent this?

EDIT: In the generated HTML, I see this:

<a href="a1-'a1'-Cama&amp;#241;o?sort=estadisticas#241;o">a1 'a1' Cama&amp;#241;o</a>

Later in the page I have this, and this time the display is correct:

<b>a1 'a1' Cama&#241;o</b>

The first is generated this way:

<%= Html.RouteLink(Html.Encode(Model.NAME),  ...... %>

and the second like this:

<%= Html.Encode(Model.NAME)%>

So my guess is that the problem is with the Html.RouteLink.

From stackoverflow

• Are you encoding twice accidentally?

  For example, if you set the Textarea's content programmatically on the server side, it will encode the content automatically on render.

  Try looking at the raw HTML output of the textarea.

  Normally when you put escapes inside textarea content, it should shows up in the textarea decoded (displayed as the intended unescaped character).

  So it might be a problem of accidentally Html.Encode twice unnescessarily.

  If your data is already escaped, you might want to un-escape (Html.Decode) it before putting it in the textarea.

  Tomalak : Same thought here. +1

  chakrit : Saw it. Still apply, no?

• So my guess is that the problem is with the Html.RouteLink

  Yep. You're not supposed to HTML-encode the parameter going into RouteLink, it generates the HTML itself and so will take care of escaping for you.

Howto rewrite phone numbers in TSQL?

I have a beginners question in TSQL.

We imported Excel Sheets into a SQL Server 2008. Too bad these excel files were not formatted the way they should be. We want a phone number to look like this: '012345678', no leading and trailing whitespace and no whitespace within. Even worse sometimes the number is encoded with a prefix '0123-2349823' or '0123/2349823'.

Normally I would export the excel file to csv, then launch some magic perl script to do the cleaning and then reimport the excel file.

Still it would be interesting to know how to do stuff like this with TSQL.

Any ideas?

From stackoverflow

• Here is a good article pertaining to SQL Server.

  : Thanks, thats an interesting article but doesn't solve the issue. Guess that whole regex, substring thingy is way easier to do in perl than in tsql.

• "Cleaned" contains only numeric value

  Depending on whether a telephone number contains "-", "/", replace them with an empty string.

  create table #t ( tel varchar(30) )
  
  insert  #t select '0123-2349823' 
  insert  #t select '0123/2349823'
  
  select  tel,
       replace(tel, 
        case
         when patindex('%-%', tel) > 0 then '-'
         when patindex('%/%', tel) > 0 then '/'
        end, '') as Cleaned
  from    #t
  

  : What is #t? Never seen that one before.

  Sung Meister : It's a temporary table I used to test so that you can simply copy and paste the code above to see if it works in your environment

• Something like

  replace(replace(rtrim(ltrim('0123-2349823')), '-', ''), '/', '')
  

  should work. Doesn't look pretty. ;)

  : That was easy :)

  Sung Meister : +1: Ha, i overthunk. This looks better than what I suggested. ;)

  Pawel Krakowiak : @nooomi: You said you like Perl... :P

• I would go about it with an update and use the 'Replace' and LTrim/RTrim functions for SQL.

  Update Table1
  set phonenum = Case
       When phonenum like '%-%' Then LTrim(RTrim(Replace(phonenum, '-', '')))
        Else LTrim(RTrim(Replace(phonenum, '/', '')))
       End
  

.NET: Animation common control for .NET?

Did Microsoft not provide a managed wrapper around the Animation common control for .NET developers?

From stackoverflow

• No, but you can get it here.

  Chris Pietschmann : Beware the solution linked to doesn't work in Windows 7 Beta.

  Ian Boyd : Is the Animation common control removed in Windows 7? Or does the random code that someone found on the internet not work?

Unexpected result from HttpURLConnection - reading remote binary file.

I'm trying to read a remote binary file (say, image) from internet like this:

HttpURLConnection connection = (HttpURLConnection) myUrl.openConnection(); //myUrl - URL object pointing for some location
if(connection.getResponseCode() == 200){
    File temp = File.createTempFile("blabla", fileName); //fileName - string name of file
    FileOutputStream out = new FileOutputStream(temp);
    int fileSize = Integer.parseInt(connection.getHeaderField("content-length"));
    int counter = 0;
    DataInputStream in = new DataInputStream(connection.getInputStream());
    byte ch[] = new byte[1024];
    System.out.println(counter);
    while((counter += in.read(ch)) > 0){
        out.write(ch);
        if(counter == fileSize){
            out.close();
            break;
        }
    }
}

Locally of with local web server (localhost) it works perfectly.

But. Then myUrl is URL of file on some remote web server - it returns unexpected results. For instance, from sources of given files it seems that it repeats some packages (I think because of corruption of previous ones or someting) and the resulting file usually is about 10% bigger than original one because of this repeats. So file is corrupted and cannot be opened correctly with image viewers.

How can I solve this?

From stackoverflow

• read does not necessarily read the entire buffer (particularly if it is at the end of the stream).

  So change your loop:

  for (;;) {
      int len = in.read(ch);
      if (len == -1) {
          break;
      }
      out.write(ch, 0, len);
  }
  

  Perhaps put that code in a method somewhere.

  Also note:

  • There is no point in using DataInputStream here (although readFully is often useful).

  • Always close resource (such as streams) with the usual idiom:

    final Resource resource = acquire(); try { use(resource); } finally { resource.close(); }

  • Probably wont make much difference, but a buffer size of 1024 is a bit small. I tend to default to 8192 arbitrarily.
  Peter Walser : Just for completion, most Java programmers prefer the shorter variant: int len; while ((len = in.read(ch)) >= 0) { out.write(ch, 0, len); } Saves one condition, a break and reuses the variable on the stack, which makes it a little less error prone.

  Tom Hawtin - tackline : A prefer to avoid side-effects. Reusing variables is not best-practice!

• Oh, just minute after asking found solution on another thread.

  Thank you, Tom, for your time and absolutely fast reply.

[Python] Is there anything that cannot appear inside parentheses?

I was intrigued by this answer to my question about getting vim to highlight unmatched brackets in python code. Specifically, I'm talking about the second part of his answer where he mentions that the C syntax highlighting is actually flagging as an error any instance of curly braces inside parens. It is an unobtrusive cue that you have unclosed parens when all of your downstream curly braces light up in red.

That trick works because C syntax doesn't allow curly braces inside parentheses. To satisfy my (morbid?) curiosity, can I do something similar with python code? Is there anything in python syntax that isn't legal inside parentheses?

Note: I'm not trolling for a better answer to my other question (there are plenty of good answers there already). I'm merely curious if this trick is even possible with python code.

From stackoverflow

• I'm not sure what are you trying to do, but how about "def" or "class"?

  this snippet is valid when it's not inside parenthesis

  class dummy: pass
  

  S.Lott : No Statement can be ()'d Only expressions.

• Any Python statement (import, if, for, while, def, class etc.) cannot be in the parentheses:

  In [1]: (import sys)
  ------------------------------------------------------------
  File "<ipython console>", line 1
    (import sys)
         ^
  <type 'exceptions.SyntaxError'>: invalid syntax
  

  Richard Levasseur : Does this include the default if-else? a = b if c else d?

  Andrew Dalke : the "b if c else d" can be in ()s because it's an expression. The assignment part ("a = ...") cannot, because in Python assignment is a statement. In Python, no statement can be in parentheses.

• Here's an exact answer:

  • http://docs.python.org/reference/expressions.html#grammar-token-expression_list
  • http://docs.python.org/reference/compound_stmts.html#function

How to create a WPF Window without a border that can be resized via a grip only?

If you set ResizeMode="CanResizeWithGrip" on a WPF Window then a resize grip is shown in the lower right corner, as below:

If you set WindowStyle="None" as well the title bar disappears but the grey bevelled edge remains until you set ResizeMode="NoResize". Unfortunately, with this combination of properties set, the resize grip also disappears.

I have overridden the Window's ControlTemplate via a custom Style. I want to specify the border of the window myself, and I don't need users to be able to resize the window from all four sides, but I do need a resize grip.

Can someone detail a simple way to meet all of these criteria?

1. Do not have a border on the Window apart from the one I specify myself in a ControlTemplate.
2. Do have a working resize grip in the lower right corner.
3. Do not have a title bar.

From stackoverflow

• If you set the AllowsTransparency property on the Window (even without setting any transparency values) the border disappears and you can only resize via the grip.

  <Window
      xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
      xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
      Width="640" Height="480" 
      WindowStyle="None"
      AllowsTransparency="True"
      ResizeMode="CanResizeWithGrip">
  
      <!-- Content -->
  
  </Window>
  

  Result looks like:

  

  ZombieSheep : Pure fluke I knew this - I was playing with the same control set myself this afternoon. :)

  Tomáš Kafka : Wow, I wouldn't expect this, but it is totally handy for make-your-own-post-it-notes-in-5-minutes, thanks :)

How do I code a rake task that runs the Rails db:migrate task?

I would like to run db:migrate VERSION=0 and then db:migrate inside of my own rake task. I am confused about how to do this. Do I need a special require statement? My rake task will reside in the lib/tasks directory of a Rails app. Thanks.

From stackoverflow

• EDIT: Rake::Task[] won't accept parameters, you have to set it in ENV. In addition, you have to reenable the task to run it multiple times.

  ENV['VERSION']= '0'
  Rake::Task['db:migrate'].invoke
  Rake::Task['db:migrate'].reenable
  ENV.delete 'VERSION'
  Rake::Task["db:migrate"].invoke
  

  NOTE: Rake::Task.reenable requires Rake 0.8.2 or higher.

  fooledbyprimes : I tried this inside of my custom rake task (residing in lib/tasks) but it failed. ("rake aborted Don't know how to build task db:migrate VERSION=0")

• Check out rake db:reset as that will accomplish what you are trying to do.

  To see what all of your rake tasks do, run rake -T

  fooledbyprimes : This is interesting but it does not really explain how to run a pre-built rails task inside of a newly created custom rake task.

  erik : Yep, since Pesto got his answer in before mine, I figured it wouldn't hurt to show you an easier way even though it didn't answer the question exactly.

  fooledbyprimes : Actually Pesto's answer didn't work.

• Is your task just dependent on having a clean db? If that's the case then you can do:

  task :my_task => [:environment, 'db:reset']

  fooledbyprimes : Okay I like this. It works. This must surely mean that the :environment task loads all the rails rake namespaces.

LINQ joining two tables

I have two tables say A and B. A cols are GUID, someintVar, someMoreIntvar B col are GUID, someItemNO, SomeItemDesc

Now for one GUID I will have only one row in Table A. But I can have multiple rows for the same GUID. Now I want to query the database based on GUID and select values in a class. This class will have a list that will hold different rows coming from the second table. How can I do it?

Right Now I am getting many items in the result based on how many rows are there in the second table for that GUID.

var itemColl = from p in db.A
               join item in db.B on p.CardID equals item.CardID
               where p.CardID == "some GUID"
               select new 
               {
                   p.CardID,
                   p.secondCol,
                   p.ThirdCol,
                   item.ItemNo // How to add them in a collection or list.
               };

From stackoverflow

• Unested, but how about re-writing it a bit:

  var itemColl = from p in db.A
                 where p.CardID == "some GUID"
                 select new {
                     p.CardID,
                     p.secondCol,
                     p.ThirdCol,
                     Items = db.B.Where(b=>b.CardID==p.CardID)
                        //.Select(b=>b.ItemNo) [see comments]
                 }
  

  Alternatively, you could perhaps group...

  Tanmoy : Thanks it is working. But just one thing. Its returning a list but not of type List. How can I convert it? Actually I am having a class with a List property. I need to select this class object.

  Tanmoy : It is returning System.Collections.Generic.List I need it to have List

  John Boker : just remove the .Select(b=>b.ItemNo) and the type will be List

  Marc Gravell : Sorry, I based that on the "item.ItemNo" in the final Select - just remove the Select

  Tanmoy : Got it. :) Thanks a lot

• Assuming this is happening in the NEW or LOAD method of your class...this is how I would do it...

  dim myAItem AS A = (from x in db.As WHERE x.CardID == MyGUIDValue).SelectSingleOrDefault
  
  ' Assign Variables Here
  Me.AValue1 = myAItem.FromDbValue1
  
  dim itemColl = (from b in db.Bs on b == MyGUIDValue).ToList 
  
  me.ItemList = New List(of MySubClass)
  For each bItem as B in itemColl
     dim item as New MySubClass
     'Assign Variables Here, ex: item.Value1 = bItem.MyDbValue1
     me.ItemList.Add(item)
  Next
  

• Assuming you have a foreign-key relationship set up between A and B on GUID. (And if you don't you db schema is broken and needs to be fixed)

  var itemColl = from p in db.A
                 where p.CardID == "some GUID"
                 select new 
                 {
                     p.CardID,
                     p.secondCol,
                     p.ThirdCol,
                     Items = p.Bs
                 }
  

Appropriate placement for my testcases belonging to a non-app in Django

I have built my website in Django. And like any other django project I have got apps inside the project root directory and some special folders like(extensions a.k.a custom django command extensions). With an app, we dont have any problem with having testcases. "tests.py" inside the app directory will be the solution. But for a special folder(which is not an app or which doesn't have a models.py) where do I place the testcases. I tried placing the tests.py inside the extensions directory and it said the directory is not a model and unable to run the tests. HOw do I solve this? How can I have a proper placement of testcases related to non-apps?

From stackoverflow

• I think it will work to put them in a tests/ directory at the project level.

  If, for some reason, it doesn't then try creating an empty models.py in your extensions directory.

  Maddy : Empty models.py will solve the issue but its not a good practice to have a file which does nothing right! And tests/ directory at the project level should hold project level tests. My case is evaluating only certain tests related to a particular directory(extensions in my case).

  Van Gale : Needing an empty models file is a standard Django problem/feature :) It's because of the way Django tests for app directories, just as Python itself needs an __init__.py file for packages even if the file is empty.

  Carl Meyer : Yes, empty models.py is the right solution. It doesn't "do nothing" - it says "this is a Django app." Same way an empty __init__.py says "this is a Python package." Nothing wrong with that.

I want to run an scons command in my current directory. How do I do this in Java?

I am using Java to automate a build. I would like to run an 'scons' command in the current directory in which the build is being performed. How can I do this with Java? I have tried

Runtime.getRuntime().exec(CurrentDir + commandString)

This didn't work. Any ideas would be greatly appreciated.

From stackoverflow

• You must use the Java 5 API ProcessBuilder to change the current directory for a new process.

MySQL Type for Storing a Year: Smallint or Varchar or Date?

I will be storing a year in a MySQL table: Is it better to store this as a smallint or varchar? I figure that since it's not a full date, that the date format shouldn't be an answer but I'll include that as well.

Smallint? varchar(4)? date? something else?

Examples:

• 2008

• 1992

• 2053

From stackoverflow

• I would use the YEAR(4) column type... but only if the years expected are within the range 1901 and 2155... otherwise, see Gambrinus's answer.

  Nerdling : Well I feel silly for having completely missed this type. Thanks

  R. Bemrose : It's not a widely used type. The only advantage it has over the smaller int types is that you can use date functions on it.

  Paul Dixon : As it's only one byte wide, it only supports years in range '1901' to '2155'

  R. Bemrose : @Paul: That's true... I've noted that in my answer now.

  Nerdling : @Paul: I'd believe that MySQL would be updated before then to avoid a Y2.155K bug — I hope. Good concern either way.

  Gambrinus : thanks for mentioning the year-datatype - didn't know that one

  nickf : Yeah thanks for the tip - I'd never heard of this one either!

• I'd go for small-int - as far as I know - varchar would take more space as well as date. second option would be the date.

• My own experience is with Oracle, which does not have a YEAR data type, but I have always tried to avoid using numeric data types for elements just because they are comprised only of digits. (So this includes phone numbers, social security numbers, zip codes as well, as additional examples).

  My own rule of thumb is to consider what the data is used for. If you will perform mathematical operations on it then store it as a number. If you will perform string functions (eg. "Take the last four characters of the SSN" or "Display the phone number as (XXX) XXX-XXXX") then it's a string.

Hibernate: load a field from a query , but don't insert it to the table

Hey

I have a bean with a certain field f1 that should not be mapped into the table , but sometime I do want to load it from some queries (not the table itself)

Can it be done? How?

I've tried declaring it @Transient , but then it doesn't read it from the query , even when I declare <return-property name="f1" column="f1"/>

Thanks!

From stackoverflow

• Try mapping it like normal but setting insert and update to false. Or, you could define it as a formula that just has the column name in the formula. Hibernate will query it just fine but will know not to try to write it.

  yossale : Insertable = false and updateable = false will still cause hibernate to try to read it from the table - and I don't want it. (Also , there's no readable = false) About the formula - Good idea , but it won't work - because the result is based on other fields in the query , which are not in the table

  Brian Deterling : I guess I misunderstood the question. You might try adding more details or a concrete example. Good luck!

Async process start and wait for it to finish

I am new to the thread model in .net. What would you use to:

1. start a process that handles a file (process.StartInfo.FileName = fileName;)
2. wait for the user to close the process OR abandon the thread after some time
3. if the user closed the process, delete the file

Starting the process and waiting should be done on a different thread than the main thread, because this operation should not affect the application.

Example:

My application produces an html report. The user can right click somewhere and say "View Report" - now I retrieve the report contents in a temporary file and launch the process that handles html files i.e. the default browser. The problem is that I cannot cleanup, i.e. delete the temp file.

From stackoverflow

• "and waiting must be async" - I'm not trying to be funny, but isn't that a contradiction in terms? However, since you are starting a Process, the Exited event may help:

          ProcessStartInfo startInfo = null;
          Process process = Process.Start(startInfo);
          process.EnableRaisingEvents = true;
          process.Exited += delegate {/* clean up*/};
  

  If you want to actually wait (timeout etc), then:

  if(process.WaitForExit(timeout)) {
      // user exited
  } else {
      // timeout (perhaps process.Kill();)
  }
  

  For waiting async, perhaps just use a different thread?

          ThreadPool.QueueUserWorkItem(delegate {
              Process process = Process.Start(startInfo);
              if(process.WaitForExit(timeout)) {
                  // user exited
              } else {
                  // timeout
              }
          });
  

• Try the following code.

  public void KickOffProcess(string filePath) {
    var proc = Process.Start(filePath);
    ThreadPool.QueueUserWorkItem(new WaitCallBack(WaitForProc), proc);
  }
  
  private void WaitForProc(object obj) {
    var proc = (Process)obj;
    proc.WaitForExit();
    // Do the file deletion here
  }
  

  Marc Gravell : Not sure it handles the "OR abandon the thread after some time"

• I would probably not use a separate process for opening a file. Instead, I'd probably utilize a background thread (if I thought the operation was going to take a long time and possible block the UI thread).

  private delegate void FileOpenDelegate(string filename);
  
  public void OpenFile(string filename)
  {
     FileOpenDelegate fileOpenDelegate = OpenFileAsync;
     AsyncCallback callback = AsyncCompleteMethod;
     fileOpenDelegate.BeginInvoke(filename, callback, state);
  }
  
  private void OpenFileAsync(string filename)
  {
     // file opening code here, and then do whatever with the file
  }
  

  Of course, this is not a good working example (it returns nothing) and I haven't shown how the UI gets updated (you have to use BeginInvoke at the UI level because a background thread cannot update the UI thread). But this approach is generally how I go about handling asynchronous operations in .Net.

  Bogdan Gavril : Sorry, actually I meant that the process should handle the file, not open it. I rewrote the question.

• You can use the Exited event in Process class

  ProcessStartInfo info = new ProcessStartInfo();
  
  info.FileName = "notepad.exe";
  Process process = Process.Start(info);
  
  process.Exited += new EventHandler(process_Exited);
  Console.Read();
  

  and in that event you can handle the operations you mentioned

  Bogdan Gavril : this isn't async

SQL MAX of column including its primary key

Short:

From below sql select I get the cart_id and the value of the maximum valued item in that cart.

select CartItems.cart_id, max(ItemValues.value)
from CartItems inner join ItemValues on CartItems.item_id=ItemValues.item_id
group by CartItems.cart_id

but I also need item_id for that item (ItemValues.item-id).

Long:

Two tables, CartItems, ItemValues (and their respective Carts, Items, irrelevant here).
Each cart can have several items wheras each item has one value defined in ItemValues.
Each item belongs to one cart.
The value of a cart is the value of the item with maximum value within its cart.
How do I select cart-id, max(item-value) and it's corresponding item-id?

For instance cart-id A contains item-id X with value 10 and item-id Y with value 90.
With above sql select I get,

A, 90

What I need is

A, Y, 90

platform: MS SQL

From stackoverflow

• In MS SQL and Oracle:

  SELECT *
  FROM
    (
    SELECT ci.*, iv.*, ROW_NUMBER() OVER (PARTITION BY CartItems.cart_id ORDER BY ItemValues.value DESC)
    FROM   CartItems ci
    INNER JOIN ItemValues iv
       ON CartItems.item_id=ItemValues.item_id
    ) s
  WHERE rn = 1
  

  In MySQL:

  SELECT
  FROM
    (
    SELECT ci.*,
           (
           SELECT id
           FROM ItemValues iv
           WHERE iv.item_id = ci.item_id
           ORDER BY
                 value DESC
           LIMIT 1
           ) AS maxitem
    FROM   CartItems ci
    ) iv, ItemValues ivo
  WHERE ivo.id = iv.maxitem
  

  Dent Argue : Thanks! With a little modification of your first select I now get cart-id and its respective item-id. I guess the results then have to be joined with the values table to get the actual value too. Right? Anyways, thanks a lot! It also opened my eyes for partion by etc :)

  Quassnoi : No, you don't have to join, I just missed ItemValues.* in the first subquery. See updated post.

  Dent Argue : Great stuff, thanks!

  Byron Whitlock : Thank you! Works like a charm!

How do you login to a webpage and retrieve its content in C#?

How do you login to a webpage and retrieve its content in C#?

From stackoverflow

• Look at System.Net.WebClient, or for more advanced requirements System.Net.HttpWebRequest/System.Net.HttpWebResponse.

  As for actually applying these: you'll have to study the html source of each page you want to scrape in order to learn exactly what Http requests it's expecting.

• That depends on what's required to log in. You could use a webclient to send the login credentials to the server's login page (via whatever method is required, GET or POST), but that wouldn't persist a cookie. There is a way to get a webclient to handle cookies, so you could just POST the login info to the server, then request the page you want with the same webclient, then do whatever you want with the page.

• Use the WebClient class.

  Dim Html As String
  
  Using Client As New System.Net.WebClient()
      Html = Client.DownloadString("http://www.google.com")
  End Using
  

  Slee : I did not know about DownloadString - awesome - thanks!

  Joel Coehoorn : Why was this downvoted?

  JohnFx : He asked for C# code, probably (it wasn't me that downvoted it)

  Joel Coehoorn : The C# is almost identical- just a parenthese, braces, a semi-colon, and change the case of a few keywords.

  Josh Stodola : Come on, people. If you can read/write C#, you can read/write VB. Open your mind!

  Alex Fort : I downvoted because he mentioned that he needs to login before he downloads the page. The stock webclient wouldn't support cookies for a login session, so the solution didn't exactly fit the problem.

  Josh Stodola : It could work with cookieless sessions, I guess.

  JohnFx : The C# answer (mine) got downvoted too. I suppose it was the authentication thing.

• How do you mean "login"?

  If the subfolder is protected on the OS level, and the browser pops of a login dialog when you go there, you will need to set the Credentials property on the HttpWebRequest.

  If the website has it's own cookie-based membership/login system, you will have to use HttpWebRequest to first response to the login form.

• Try this:

  public string GetContent(string url)  
  { 
    using (System.Net.WebClient client =new System.Net.WebClient()) 
    { 
    return client.DownloadString(url); 
    } 
  }
  

• You can use the build in WebClient Object instead of crating the request yourself.

  WebClient wc = new WebClient();
  wc.Credentials = new NetworkCredential("username", "password");
  string url = "http://foo.com";   
  try
  {
   using (Stream stream = wc.OpenRead(new Uri(url)))
   {
    using (StreamReader reader = new StreamReader(stream))
       {
           return reader.ReadToEnd();
               }
   }
  }
  catch (WebException e)
  {
   //Error handeling
  }
  

• string postData = "userid=ducon";
              postData += "&username=camarche" ;
              byte[] data = Encoding.ASCII.GetBytes(postData);
              WebRequest req = WebRequest.Create(
                  URL);
              req.Method = "POST";
              req.ContentType = "application/x-www-form-urlencoded";
              req.ContentLength = data.Length;
              Stream newStream = req.GetRequestStream();
              newStream.Write(data, 0, data.Length);
              newStream.Close();
              StreamReader reader = new StreamReader(req.GetResponse().GetResponseStream(), System.Text.Encoding.GetEncoding("iso-8859-1"));
              string coco = reader.ReadToEnd();
  

Determining if a lat-long circle and a circle on a sphere overlap

I am creating an application with Geo Proximity Search capability using PHP as server scripting language and MySQL as Databse.

Consider a situation:

Where we have certain set of objects having latitude and longitude positions assosciated with respective objects. On specifying location details like country and/or city along with range/radius in KM we are getting objects that are lying withing that radius/range using MySQL query for example:

"SELECT [columns] FROM [column] WHERE 1=1 AND 3963.191 * ACOS((SIN(PI() * [latitude] / 180) * SIN(PI() * [column].latitude / 180)) + (COS(PI() * [latitude] /180) * cos(PI() * [column].latitude / 180) * COS(PI() * [column].longitude / 180 - PI() * [longitude] / 180)) ) <= 10"

Above calculations will give the objects that are within the 10 KM area from the country/city center point on earth.

Now suppose i have one object (with latitude and longitude) which is not in this 10 KM area, but for example 15 KM from the country/city center point on earth. This object has service range 10 KM (giving service upto 10 KM area).

Now the question is if i am looking for the objects within the 10 KM range from country/city center point then the object (15 KM apart from country/city center point) having service range 10 KM should also be included in search.

How can i make it possible? I have country/city center lat, long coordinates,range/radius(in which we have to find out objects) and object having delivery radius (for e.g.10 KM) and its corresponding coordinates.

Can you guide me how to do that using PHP, MySQL?

From stackoverflow

• Use MySQL spatial extensions http://dev.mysql.com/doc/refman/5.1/en/spatial-extensions.html

  On the other hand, if you want just to identify other circles that intersect with given one, they would be all of these, for which distance between centers of the circle is less then sum of radius. In other words, assuming that your original point and range is given by triple (x₀,y₀,r₀), you need all (x_n,y_n,r_n) for which

  √((x₀ - x_n)² + (y₀ - y_n)²) ≤ r₀ + r_n

  Asif : This may require change in existing database schema. Is there any other method?

  vartec : old school geometry. I've added the formula.

  Asif : Hi Vartec, Thanx for prompt reply. I have edited my question so you can provide me better solution is such scenario. Thanx in advance.

  vartec : Well, basically you already have the left side, and in the right side in your formula you only have r₀. You'd want to add r_n ([column].range?). In other words, instead of <= 10, <= 10 + [column].range. If range is always 10km, well the just <= 20.

Defuzzification process of fuzzy sets in RDBMS

I have a table that stores all values..eg x1, x2, x3 which determines fuzzy sets. Degree of membership is assigned to each using mathematical functions for Low, Med, High.

My rule 1 states that if x1 is high AND x2 is medium then probability of output is z. I then take min{x1,x2} to evaluate the rule. Rule 2 states that if x2 is high OR x3 is medium then output is max{x2,x3}.

Now to defuzzify I take aggregation of rule consequences to find out the output of the 2 rules. I have all degrees of membership defined (0 to 1) for x1 x2 x3 for each rule.

How do I defuzzify?

From stackoverflow

• Okay, first of all, are you using a probabilistic logic or a fuzzy logic. While similar, they're not identical. If you're really modeling probabilities here, then you need to look into this via Bayes Theorem as a conditional probability.

  If these are really fuzzy truth values, then you need to have a model of set-membership, which we need to know.

  CGF : The sets are fuzzy and Bayesian reasoning won't be applied.. Fuzzy output is defined as the degree of membership to which they belong[not the probability as output].I forgot that the consequent of the rule could have multiple parts. This was the missing link! Now ok! The problem is now solved -

  Charlie Martin : Yay! Haven't done fuzzy systems in a long time, fun to think back to them.

Rails Menus

I would like to know if there is a good plugin for rendering navigation links in Rails. It would be nice if it could render links to all possible GET actions on a controller directly from the routes file.

From stackoverflow

• Can I ask the purpose of your question? There are only four get actions in a RESTful controller, (index, show, edit, and new). The overhead in producing a list using a special route generator on the fly is probably too much.

  You could simply create a partial that can render those four actions for whatever controller you're currently viewing (think params[:controller]).

  As far as for all possible Get actions: All possible get actions would encompass the show action for each item in your database. That is, again, best handled in a partial and the use of link_to.

  It's hard to give a complete answer though, because your circumstances seem unique.

  I hope this helps even a little.

  -Chris

  Chris Drappier : a restful controller can have n number of actions, it's only rails convention that enumerate the 4 actions you mentioned. some resources require more representations. I.E. map.resources :elements, :member => [:summary, :dots, :issues, :link_to_irma_organizations, :lists, :manage_permissions, :users]

  Chris Drappier : So, my current method of link rendering is a yml configuration file that enumerates the links to the get actions for each controller, this can very easily get out of sync with the available routes because it has to be manually edited. I'd much rather load the menu configuration from the routes.

  Christopher Hazlett : Sorry. Didn't realize you commented. I hear what you're saying, the standard configuration probably mandates that you separate your concerns. you've got a manage_permissions and a users method for a controller. Seems like you need to split your controller up into more distinct concerns.

  Christopher Hazlett : Also, using YAML as a configuration method for your menu is probably a bad idea. YAML is not going to scale well. If you're putting data into a YAML file, you might just consider building a menu model and loading that on each call. MySQL will outperform YAML in the long run.

  Chris Drappier : The yaml file is loaded into a constant when the application is loaded,I could do the same with the database, i guess, but it's really nice to have all the configuration in a file that can be changed quickly

  Chris Drappier : the actions in the route configuration do nothing but render templates from the views directory. the links and forms inside those templates are routed to their proper controller.

  Christopher Hazlett : It's a tough call, but I like storing menus in the database. That way I don't have to restart the WS if I change the menu structure...additionally, if you want to expose (or hide) menus according to a permission structure, that's easily accomplished with a db...and the logic is changed very easily.

• While this does not answer ths specific question, you can see all of your OWN routes on the command line by running

  rake routes
  

  This will give you your own personal readout of all of your routes on the site, however like Chris said above, this isn't really a specific answer, more of an FYI.

  Cheers!

• You should try Mmmenu: http://github.com/snitko/mmmenu It's very flexible.

How to retrieve the total row count of a query with TOP

I have a SQL Server 2008 query

SELECT TOP 10 *
FROM T
WHERE ...
ORDER BY ...

I'd like to get also the total number of the rows. The obious way is to make a second query

SELECT COUNT(*)
FROM T
WHERE ...
ORDER BY ...

Is there an efficient method?

Thanks

From stackoverflow

• Remove the ORDER BY clause from the 2nd query as well.

• No.

  SQL Server doesn't keep COUNT(*) in metadata like MyISAM, it calculates it every time.

  UPDATE: If you need an estimate, you can use statistics metadata:

  SELECT  rows
  FROM    dbo.sysindexes
  WHERE   name = @primary_key,
  

  where @primary_key is your table's primary key name.

  This will return the COUNT(*) from last statistics update.

  John : are you saying in mysql, you can do "SELECT * FROM t_user WHERE...LIMIT 0,20" and there's meta data for the total number of rows in memory? So that I don't have to make a second query with COUNT()?

  John : OH, I just answered my own question...SQL_CALC_FOUND_ROWS and FOUND_ROWS() does the job..thanks!

• Do you want a second query?

  SELECT TOP 10
      *, foo.bar
  FROM
      T
      CROSS JOIN
      (SELECT COUNT(*) AS bar FROM T WHERE ...) foo
  WHERE
      ...
  ORDER BY
      ...
  

  OR

  DECLARE @bar int
  SELECT @bar = COUNT(*) AS bar FROM T WHERE ...
  SELECT TOP 10
      *, @bar
  FROM
      T
      CROSS JOIN
      (SELECT COUNT(*) AS bar FROM T WHERE ...) foo
  WHERE
      ...
  ORDER BY
      ...
  

  Or (Edit: using WITH)

  WITH cTotal AS
  (
      SELECT COUNT(*) AS bar FROM T WHERE ...)
  )
  SELECT TOP 10
      *, cTotal .bar
  FROM
      T
  WHERE
      ...
  ORDER BY
      ...
  

  gbn : Examples 1 and 3 are single queries and functionally identical.

  gbn : You can't. They are 2 different query constructs. The COUNT will actually be quite efficient, probably more so than the main query because it will use the most efficient

• SELECT     TOP (2) *,
             (SELECT COUNT(*) AS Expr1 FROM T) AS C
  FROM         T
  

  gbn : Would this not do "SELECT COUNT(*) AS Expr1 FROM T" for every output row? In this case 10 rows

  Mufasa : I haven't profiled this, but I think this would actually be slower since it is in a sub-query and returned in each row. I'm no guru though, the server might be smart enough to cache it. Even still, I don't think it would be any *faster* than two queries since it is still running them both.

WebService, WebMethod and Inheritance

In a Web Service context, I have the following class which inherit from the class Mammal. The Mammal class is defined in a proxy. I cannot change the definition of that class. Because I need to add some methods to the class Mammal on the client side, I inherited Mammal and created Giraffe.

namespace TestApplication
{  
    public class Giraffe : Mammal
    {
        public Giraffe()
        {
        }
    }
}

When I call a WebMethod which expect an object of type Mammal, I get the following exception telling me that Giraffe isn't expected.

  Error: System.InvalidOperationException: There was an error generating the XML document. ---> System.InvalidOperationException: The type Giraffe was not expected. Use the XmlInclude or SoapInclude attribute to specify types that are not known statically.
   at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationWriterPaymentRequestAuthorization.Write6_Tender(String n, String ns, Tender o, Boolean isNullable, Boolean needType)
   at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationWriterPaymentRequestAuthorization.Write12_PaymentRequestAuthorization(String n, String ns, PaymentRequestAuthorization o, Boolean isNullable, Boolean needType)
   at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationWriterPaymentRequestAuthorization.Write13_PaymentRequestAuthorization(Object o)
   --- End of inner exception stack trace ---

Is there a workaround that? I cannot really add the XmlInclude...

Thanks!

From stackoverflow

• To add methods, you should be using partial classes. Try adding a second class file with (in the right namespace etc):

  partial class Mammal {
      public void ExtraMethod() {...}
  }
  

  The partial keyword tells the compiler to allow the class to be split over multiple files - ideal for code-generation scenarios. It does, however, demand the partial keyword in every file; the members are simply combined (except for partial methods in C# 3.0 which have their own subtle behaviour). I hope that wsdl.exe (or whatever) includes this!

  Martin : I am afraid I cannot use a Partial class in this case. How do I change the class on the server?

  Marc Gravell : Out of curiosity, why not? In particular, proxies aren't at the server... what is it you are actually trying to do? So far your description is roughly "add methods to the client-side web-service proxy"; partial classes is the correct answer for that scenario...

• You have to use XmlInclude. It's your only choice. You have to tell the serializer what class you'll be serializing, since it has to generate the code to do the serialization.

  It just struck me while writing this, that you might get away with having Giraffe implement IXmlSerializable, but that's even more work.

Accounting for leap year in comparing year to year sales.

I am writing a program that shows the current years sales from the beginning of the fiscal year to the current date, compared to the same date range of the year before.

My question is, what efforts do I need to take for leap year?

UPDATE:

OK they want it like I said (compare last year up to same date) but if today is non leap year and last year is and today is feb 28th compare to last year up to 29th. Or if today is Feb 29th compare to last year up to 28th.

From stackoverflow

• Surely that depends on what the business wants you to do. Isn't this a question that should be answered by an accountant?

• This strikes me as a business decision. Depending on the type of business, that extra day may not matter. Otherwise, I suppose you could treat is as "first n days of the year" rather than "Jan 1 through X".

  John Isaacks : If I did that tho and only compared last years up to this years at the same current days at the end of the year it would be off bc this year there was 365 days and last year there were 366, so if today was the 365 day, the year would be over but it wouldn't be comparing to the total year before.

  TheTXI : @unknown: if that is the case, you could compare the two years together, but then "subtract" out the sales for February 29th and show them separately. That way you could have a fairly accurate 365-365 comparison.

  Pesto : @unknown: Which is why this is a business decision. We're happy to point out the things that need to be figured out, but ultimately you guys are going to have to pick an implementation.

  John Isaacks : @pesto OK I am going to get more requirements info from the man in charge and then edit my post to make it more specific.

• Here's an idea, but like others have said it might be based on your specific domain.

  1. Consider 1 "normalized" year = 365.242199 days counting all the leap stuff (says google)
  2. Calculate the average sales per day in your year based on the real number of days in that year
  3. Scale it up or down to sales per 365.242199 days

  So for example

     2007 = $4000 in sales.
     There's 365 days in 2007, so avg sale per day = $10.96
     Multpiplying times num days in a normalized year (365.242199) 
        gives you $4003.05 normalized sales
  

  You can compare this directly to a similar calculation for 2008,

     2008 = $5000 in sales.
     There's 366 days in 2008, so avg sale per day = $13.66
     Multpiplying times num days in a normalized year (365.242199) 
        gives you $4975.655 normalized sales for 2008!
  

• You could scale down the leap year values to take the extra day into account.

  So if you compare, say, the 1st of September of a regular year with the 1st of September of a leap year you would do:

  if(year == leapyear && day > 28Feb)
      Convert date to dayOfYear
      leapYearValue *= dayOfYear / (dayOfYear + 1)
  

  This should really be in the specification, though.